Dual attainment and convergence of TV source certificates

Put $q^{†}=A^{\ast }{p}^{†}$. Applying [F2] at $u^{†}$ with $z=0$ and $z=2u^{†}$, and using $R(0)=0$ and $R(2u^{†})=2R(u^{†})$, gives $⟨q^{†},u^{†}{⟩}_X=R(u^{†})$. For arbitrary $z\in X$, [F2] then yields $R(z)\ge ⟨q^{†},z{⟩}_X$, so $q^{†}\in \mathrm{\partial }R(0)$ and $p^{†}\in K$. Every $p\in K$ satisfies $⟨f,p{⟩}_Y=⟨u^{†},A^{\ast }p{⟩}_X\le R(u^{†})$; equality holds for $p^{†}$. Thus $P_0$ is nonempty, closed, and convex.

The set $K$ is nonempty because $0\in \mathrm{\partial }R(0)$, convex by the subgradient inequality, and closed because $\mathrm{\partial }R(0)=\{q:R(z)\ge ⟨q,z{⟩}_X\text{ for all }z\}$ is an intersection of closed half-spaces and $A^{\ast }$ is continuous. For any nonempty closed convex subset $C$ of a Hilbert space, let $d={\inf }_{c\in C}\mathrm{\parallel }x-c\mathrm{\parallel }$ and use Countable Choice to select $c_n\in C$ with $\mathrm{\parallel }x-c_n{\mathrm{\parallel }}^2\le d^2+1\mathrm{/}n$. The midpoint belongs to $C$, so the parallelogram identity gives $\mathrm{\parallel }{c}_n-c_m{\mathrm{\parallel }}^2\le 2\mathrm{/}n+2\mathrm{/}m$; thus $(c_n)$ is Cauchy. Completeness and closedness yield a nearest point, and strict convexity of squared norm makes it unique. Applying this to $K$ and $P_0$ defines their projections.

Completing the square gives $D_{\alpha ,w}(p)=\frac{\mathrm{\parallel }f+w{\mathrm{\parallel }}_Y^2}{2\alpha }-\frac{\alpha }{2}\mathrm{\parallel }p-(f+w)\mathrm{/}\alpha {\mathrm{\parallel }}_Y^2$. Hence its unique maximizer on $K$ is $p_{\alpha ,w}={\mathrm{Proj}}_K((f+w)\mathrm{/}\alpha )$.

Let $u_{\alpha ,w}$ be a supplied global minimizer and put $p=(f+w-A{u}_{\alpha ,w})\mathrm{/}\alpha$. For any $h\in X$ and $t>0$, objective minimality and expansion of the square give $[R(u_{\alpha ,w}+th)-R(u_{\alpha ,w})]\mathrm{/}t\ge ⟨A^{\ast }p,h{⟩}_X-t\mathrm{\parallel }Ah{\mathrm{\parallel }}_Y^2\mathrm{/}(2\alpha )$. As $t\downarrow 0$, the lower limit of these secants is at least $⟨A^{\ast }p,h{⟩}_X$; convexity makes the unit secant at least every smaller positive secant, so $R(u_{\alpha ,w}+h)-R(u_{\alpha ,w})\ge ⟨A^{\ast }p,h{⟩}_X$. This for every $h$ proves $A^{\ast }p\in \mathrm{\partial }R(u_{\alpha ,w})$. Applying [F2] with $z=0$ and $z=2u_{\alpha ,w}$ gives $⟨A^{\ast }p,u_{\alpha ,w}{⟩}_X=R(u_{\alpha ,w})$, and then [F2] for arbitrary $z$ gives $A^{\ast }p\in \mathrm{\partial }R(0)$, so $p\in K$. By [F4] at $0$, $R^{\ast }(A^{\ast }q)=0$ for every $q\in K$; Fenchel–Young in [F4] therefore gives $⟨A^{\ast }q,u_{\alpha ,w}{⟩}_X\le R(u_{\alpha ,w})$. Thus $D_{\alpha ,w}(q)\le R(u_{\alpha ,w})+\alpha ⟨p,q{⟩}_Y-\alpha \mathrm{\parallel }q{\mathrm{\parallel }}_Y^2\mathrm{/}2=D_{\alpha ,w}(p)-\alpha \mathrm{\parallel }q-p{\mathrm{\parallel }}_Y^2\mathrm{/}2$. Hence $p=p_{\alpha ,w}$. Finally $A{u}_{\alpha ,w}=f+w-\alpha p$, so $\frac{1}{2}\mathrm{\parallel }A{u}_{\alpha ,w}-(f+w){\mathrm{\parallel }}_Y^2+\alpha R(u_{\alpha ,w})=\alpha D_{\alpha ,w}(p)$, establishing the normalized value identity in the Statement.

If $P_{K}x=p$ and $P_{K}y=q$, minimizing squared distance along the segment from $p$ to any $z\in K$ gives $⟨x-p,z-p⟩\le 0$. Apply this with $z=q$, and the corresponding inequality for $P_{K}y$ with $z=p$. Adding yields $\mathrm{\parallel }p-q{\mathrm{\parallel }}_Y^2\le ⟨x-y,p-q{⟩}_Y\le \mathrm{\parallel }x-y{\mathrm{\parallel }}_Y\mathrm{\parallel }p-q{\mathrm{\parallel }}_Y$ by [F7], so $\mathrm{\parallel }p-q{\mathrm{\parallel }}_Y\le \mathrm{\parallel }x-y{\mathrm{\parallel }}_Y$. Taking $x=(f+w)\mathrm{/}\alpha$ and $y=f\mathrm{/}\alpha$ proves the first noise-stability bound; applying $A^{\ast }$ proves the second.

Write $p_n=p_{{\alpha }_n,0}$. Maximality against $p_0\in P_0$ and the support bound $⟨f,p_n{⟩}_Y\le R(u^{†})$ give $\mathrm{\parallel }{p}_n{\mathrm{\parallel }}_Y\le \mathrm{\parallel }{p}_0{\mathrm{\parallel }}_Y$ and $⟨f,p_n{⟩}_Y\to R(u^{†})$. The closed span of any subsequence is separable; Gram–Schmidt on rational finite linear combinations of its terms gives a finite or countable orthonormal family with dense span. Repeated use of [F8] and [A1] selects a further subsequence whose coordinates all converge. For each finite initial segment, expanding the nonnegative squared norm after orthogonal projection shows that the sum of squared limiting coordinates is at most $\mathrm{\parallel }{p}_0{\mathrm{\parallel }}_Y^2$; hence the coordinates define a vector $p$ in that closed span. Approximation by finite linear combinations proves weak convergence there, and [F9] represents each functional $x\mapsto ⟨x,y{⟩}_Y$ restricted to the span, so convergence is weak in $Y$. The limit lies in $K$: by [F2], $K={\bigcap }_{z\in X}\{q:⟨q,Az{⟩}_Y\le R(z)\}$, an intersection of weakly closed half-spaces. It also attains $⟨f,p{⟩}_Y=R(u^{†})$, so $p\in P_0$. Weak lower semicontinuity of the norm and the bound force $\mathrm{\parallel }p{\mathrm{\parallel }}_Y=\mathrm{\parallel }{p}_0{\mathrm{\parallel }}_Y$, hence uniqueness gives $p=p_0$. Every subsequence therefore has a further subsequence weakly converging to $p_0$; testing against any fixed $y\in Y$ shows that every scalar subsequence has a further subsequence converging to $⟨p_0,y{⟩}_Y$, so $⟨p_n,y{⟩}_Y\to ⟨p_0,y{⟩}_Y$. Thus $p_n\rightharpoonup p_0$. Weak lower semicontinuity and the uniform upper bound then give $\mathrm{\parallel }{p}_n{\mathrm{\parallel }}_Y\to \mathrm{\parallel }{p}_0{\mathrm{\parallel }}_Y$. The norm identity [F10] yields strong convergence, and boundedness of $A^{\ast }$ yields strong convergence of $A^{\ast }{p}_n$.

For $u\in \mathrm{dom}R$, [F1] and [F7] give $-⟨A^{\ast }{p}^{†},u-u^{†}{⟩}_X=⟨p^{†},f-Au{⟩}_Y\le \mathrm{\parallel }{p}^{†}{\mathrm{\parallel }}_Y\mathrm{\parallel }Au-f{\mathrm{\parallel }}_Y$. Therefore the VSC [F5] holds with $\beta =0$, $p=2$, and $\psi (s)=C\sqrt{s}$, $C=1+\mathrm{\parallel }{p}^{†}{\mathrm{\parallel }}_Y$, on every regularizer sublevel; [F3] identifies its Bregman term with the distance in the Statement. Its conjugate is $H(q)=C^2\mathrm{/}(4q)$ for $q>0$. For an exact minimizer with $\mathrm{\parallel }w{\mathrm{\parallel }}_Y\le \delta$, comparison to $u^{†}$ gives $R(u_{\alpha ,w})\le R(u^{†})+{\delta }^2\mathrm{/}(2\alpha )$; if $\alpha \asymp \delta$, this is below any fixed $R_0>R(u^{†})$ for small $\delta$. With $c_p=1\mathrm{/}4$, ${\rho }_{\delta }=H(c_p\mathrm{/}\alpha )=C^2\alpha \asymp \delta$ and $H(c_p\mathrm{/}(2\alpha ))=2{\rho }_{\delta }$, so all parameter-coupling hypotheses in [F6] hold and the stated rates follow.