Cyclic exact block descent has globally optimal accumulation points

By [F1], $F$ is nonincreasing at every block update, so all cycle and within-cycle iterates remain in $S$. By [F9], $f$ is continuous, so [F4] makes $F$ lower semicontinuous; it is real-valued on $S$ because $F\le F(x^0)<+\mathrm{\infty }$. Thus [F5] bounds it below on $S$. By [F7], $L={\inf }_{k}F(x^k)$ is real; for each $\epsilon >0$, [F7] gives $j$ with $F(x^j)<L+\epsilon$, and monotonicity then gives $L\le F(x^k)\le F(x^j)<L+\epsilon$ for all $k\ge j$. Hence the full-sweep values tend to $L$.

Fix any accumulation point $\bar{x}$ of the full-sweep iterates and a subsequence $x^{k_j}\to \bar{x}$. Since $S$ is compact, [F8] makes it closed and therefore $\bar{x}\in S$. The intermediate states $y_j^0=x^{k_j},y_j^1,\dots ,y_j^n=x^{k_j+1}$ satisfy $F(x^{k_j})\ge F(y_j^i)\ge F(x^{k_j+1})$. Both endpoint values tend to $L$, so every intermediate value tends to $L$. By [A1] and continuity of the finitely many block maps, each $y_j^i$ converges to a limit ${\bar{y}}^i$, with ${\bar{y}}^0=\bar{x}$.

Fix $i$. Let $a_j$ be the coordinates other than block $i$ in $y_j^{i-1}$, and let $u_j^{-}$ and $u_j^{+}$ be its block-$i$ coordinates before and after the update. Then $a_j\to \bar{a}$, $u_j^{-}\to {\bar{u}}^{-}$, and $u_j^{+}\to {\bar{u}}^{+}$ by step 2.1; exactness gives $u_j^{+}=T_i(a_j)$. Write $m_i(a)={\min }_u\{f(a,u)+g_i(u)\}$. Since the other penalties cancel, the objective decrease is $d_j=f(a_j,u_j^{-})+g_i(u_j^{-})-m_i(a_j)$, and $d_j\to 0$ because the two intermediate objective values both tend to $L$. By [F2], $m_i(a_j)\to m_i(\bar{a})$; continuity of $f$ and lower semicontinuity of $g_i$ imply $f(\bar{a},{\bar{u}}^{-})+g_i({\bar{u}}^{-})\le m_i(\bar{a})$. The reverse inequality follows from the minimum definition, so $u^{-}=T_i(\bar{a})$ by uniqueness. Continuity of $T_i$ gives ${\bar{u}}^{+}=T_i(\bar{a})={\bar{u}}^{-}$, hence ${\bar{y}}^i={\bar{y}}^{i-1}$.

Applying step 3.1 for $i=1,\dots ,n$ shows every within-sweep limit equals $\bar{x}$ and that $\bar{x}$ minimizes every block slice. Since $\bar{x}\in S$ and $F(x^0)<+\mathrm{\infty }$, $F(\bar{x})<+\mathrm{\infty }$. By [F3], $\bar{x}$ satisfies the composite stationarity condition and is globally minimizing. Since $\bar{x}$ was arbitrary, every accumulation point is global.

The sequence lies in compact $S$, so [F6] supplies an accumulation point. If the global minimizer is unique, step 4.1 makes it the only possible accumulation point. Fix $\epsilon >0$. If infinitely many full-sweep indices lay outside the open $\epsilon$-ball about that minimizer, start with the least such index and recursively choose the least later such index; infinitude makes each candidate set nonempty, while [F10] supplies the least indices and constructs the sequence. By [F6] it has a convergent subsequence with limit in $S$, and [F11] keeps that limit outside the ball, producing a different accumulation point. Thus all sufficiently late full-sweep iterates lie in the ball. For a fixed within-sweep position, suppose a subsequence of those states converges to $y$. Its cycle-start subsequence lies in compact $S$, so [F6] supplies a further subsequence converging to some $x\in S$; steps 2.1 and 3.1 show that the corresponding within-sweep states also converge to $x$, hence $y=x$ and is a full-sweep accumulation point. Therefore every convergent subsequence of within-sweep states has the unique minimizer as its limit. If infinitely many states at that position lay outside the ball, the least-index recursion and [F6] would give a convergent subsequence whose limit remains outside by [F11], a contradiction. Hence each within-sweep sequence converges to the unique minimizer as well.