Regularized LM decreases the local solution error

For any $z\in X$ and $t\in \mathbb{R}$, expansion using [F6] gives ${\mathrm{\Phi }}_{\alpha }(h+tz)-{\mathrm{\Phi }}_{\alpha }(h)=t(⟨Th-r,Tz{⟩}_Y+\alpha ⟨h,z{⟩}_X)+\frac{t^2}{2}(\mathrm{\parallel }Tz{\mathrm{\parallel }}_Y^2+\alpha \mathrm{\parallel }z{\mathrm{\parallel }}_X^2)\mathrm{.}$ Since $h$ is a global minimizer, this is nonnegative for every $t$. Its linear coefficient must therefore vanish: if it were nonzero, a sufficiently small $t$ of the opposite sign would make the displayed difference negative. By [F3], $⟨T^{\ast }(Th-r)+\alpha h,z{⟩}_X=0$ for every $z$; taking $z=T^{\ast }(Th-r)+\alpha h$ and using positive definiteness gives $T^{\ast }(Th-r)+\alpha h=0$. Set $q:=r-Th$, so $\alpha h=T^{\ast }q$. Therefore [F3] gives $\alpha ⟨h,e{⟩}_X=⟨q,Te{⟩}_Y$, and taking the inner product of $\alpha h=T^{\ast }q$ with $h$ gives $\alpha \mathrm{\parallel }h{\mathrm{\parallel }}_X^2=⟨q,Th{⟩}_Y=⟨q,r{⟩}_Y-\mathrm{\parallel }q{\mathrm{\parallel }}_Y^2$.

Put $w:=Te+r$. By [F6] and step 1.1, $\mathrm{\parallel }e+h{\mathrm{\parallel }}_X^2-\mathrm{\parallel }e{\mathrm{\parallel }}_X^2=\frac{2}{\alpha }⟨q,w{⟩}_Y-\mathrm{\parallel }h{\mathrm{\parallel }}_X^2-\frac{2}{\alpha }\mathrm{\parallel }q{\mathrm{\parallel }}_Y^2\mathrm{.}$ Indeed, $Te=w-r$; substitute this into $2⟨e,h⟩+\mathrm{\parallel }h{\mathrm{\parallel }}^2$ and use $⟨q,r⟩=\alpha \mathrm{\parallel }h{\mathrm{\parallel }}^2+\mathrm{\parallel }q{\mathrm{\parallel }}^2$.

By [F2], $\mathrm{\parallel }q{\mathrm{\parallel }}_Y=\sigma \mathrm{\parallel }r{\mathrm{\parallel }}_Y$ and $\mathrm{\parallel }w{\mathrm{\parallel }}_Y\le (\sigma \mathrm{/}\gamma )\mathrm{\parallel }r{\mathrm{\parallel }}_Y$. Thus [F4] gives $2⟨q,w{⟩}_Y\le 2\mathrm{\mid }⟨q,w{⟩}_Y\mathrm{\mid }\le 2\mathrm{\parallel }q{\mathrm{\parallel }}_Y\mathrm{\parallel }w{\mathrm{\parallel }}_Y\le 2{\sigma }^2\mathrm{\parallel }r{\mathrm{\parallel }}_Y^2\mathrm{/}\gamma$. Substituting this and $\mathrm{\parallel }q{\mathrm{\parallel }}_Y^2={\sigma }^2\mathrm{\parallel }r{\mathrm{\parallel }}_Y^2$ into step 2.1 yields $\mathrm{\parallel }e{\mathrm{\parallel }}_X^2-\mathrm{\parallel }e+h{\mathrm{\parallel }}_X^2\ge \mathrm{\parallel }h{\mathrm{\parallel }}_X^2+\frac{2(\gamma -1){\sigma }^2}{\gamma \alpha }\mathrm{\parallel }r{\mathrm{\parallel }}_Y^2,$ which is the first claim.

Applying $T$ to $\alpha h=T^{\ast }q$ and using $q=r-Th$ gives $(T{T}^{\ast }+\alpha I_Y)q=\alpha r$. By [F8], $\alpha \mathrm{\parallel }r{\mathrm{\parallel }}_Y=\mathrm{\parallel }(T{T}^{\ast }+\alpha I_Y)q{\mathrm{\parallel }}_Y\le \mathrm{\parallel }T{T}^{\ast }q{\mathrm{\parallel }}_Y+\alpha \mathrm{\parallel }q{\mathrm{\parallel }}_Y$. The unit-ball definition [F7] gives $\mathrm{\parallel }{T}^{\ast }q{\mathrm{\parallel }}_X\le \mathrm{\parallel }{T}^{\ast }\mathrm{\parallel }\mathrm{\parallel }q{\mathrm{\parallel }}_Y$ and $\mathrm{\parallel }T(T^{\ast }q){\mathrm{\parallel }}_Y\le \mathrm{\parallel }T\mathrm{\parallel }\mathrm{\parallel }{T}^{\ast }q{\mathrm{\parallel }}_X$: if an input vector is nonzero, divide it by its norm to place it in the unit ball; if it is zero, the bound is immediate. Then [F5] gives $\mathrm{\parallel }T{T}^{\ast }q{\mathrm{\parallel }}_Y\le \mathrm{\parallel }T\mathrm{\parallel }\mathrm{\parallel }{T}^{\ast }\mathrm{\parallel }\mathrm{\parallel }q{\mathrm{\parallel }}_Y=\mathrm{\parallel }T{\mathrm{\parallel }}^2\mathrm{\parallel }q{\mathrm{\parallel }}_Y$. Thus, using $\mathrm{\parallel }T\mathrm{\parallel }\le M$ and the forcing equality, $\alpha \mathrm{\parallel }r{\mathrm{\parallel }}_Y\le (M^2+\alpha )\sigma \mathrm{\parallel }r{\mathrm{\parallel }}_Y\mathrm{.}$ Since $r\mathrm{\ne }0$, divide by $\mathrm{\parallel }r{\mathrm{\parallel }}_Y$ and rearrange using $1-\sigma >0$ to obtain $\alpha \le \sigma M^2\mathrm{/}(1-\sigma )$.