An Extended ADMM for 3-Block Nonconvex Nonseparable Problems with Applications

We consider a 3-block Alternating Direction Method of Multipliers (ADMM) for solving nonconvex nonseparable problems with a linear constraint. Inspired by \cite[Sun, Toh and Yang, $\mathit{\text{SIAM}}\mathit{\text{Journal}}\mathit{\text{on}}\mathit{\text{Optimization}}$, 25 (2015), pp.882-915]{wtwice}, the proposed ADMM follows the Block Coordinate Descent (BCD) cycle order $1\to 3\to 2\to 3$. We analyze its convergence based on the Kurdyka-Łojasiewicz property. We also discuss two useful extensions of the proposed ADMM with $2\to 3\to 1\to 3$ Gauss-Seidel BCD cycle order, and with adding a proximal term for more general nonseparable problems, respectively. Moreover, we make numerical experiments on two nonconvex problems: robust principal component analysis and nonnegative matrix completion. Results show the efficiency and outperformance of the proposed ADMM.