Joint and frozen-Newton objectives for range-invariant relaxations

Let $X,Y,Z$ be real Banach spaces, let $X_0$ be a closed linear subspace of $X$, and let $D\subseteq X$ contain $X_0$ and a nonempty set $U$. Fix $x_0\in X_0\cap U$, a map $F:D\to Y$, a bounded linear map $K:Z\to Y$, and a map $r:D\to Z$ such that $F(x)-F(x_0)=Kr(x)$ for every $x\in D$. Assume that $r{\mathrm{\mid }}_U$ is a bijection from $U$ onto a set $\tilde{U}\subseteq Z$, and that $(I-P)r^{-1}(\widehat{r})\in X_0\cap D$ for every $\widehat{r}\in \tilde{U}$ admitted below. Here $I$ is the identity on $X$ and $P:X\to X$ is a bounded linear projection with $P^2=P$, $P{\mathrm{\mid }}_{X_0}=0$, and $(I-P)(X)\subseteq X_0$. Let $\check{R}:X_0\to [0,+\mathrm{\infty }]$ be a proper convex regularizer, let $R_P:P(X)\to [0,+\mathrm{\infty }]$ be a nonnegative penalty, and let $Q:Z\times Z\to [0,+\mathrm{\infty }]$ satisfy $Q(a,b)=Q(b,a)$ and $Q(a,b)=0$ exactly when $a=b$. For $\widehat{r}\in \tilde{U}$, set

$\mathcal{R}(\widehat{r}):=\check{R}((I-P)r^{-1}(\widehat{r}))+R_P(P{r}^{-1}(\widehat{r}))\mathrm{.}$

For $p\in [1,\mathrm{\infty })$, data $y^{\delta }\in Y$, and parameters $\alpha ,\beta >0$, the joint relaxed objective is

${\mathcal{J}}_{\alpha ,\beta }^{\delta }(\widehat{r},x):=\mathrm{\parallel }K\widehat{r}+F(x_0)-y^{\delta }{\mathrm{\parallel }}_Y^p+\alpha \mathcal{R}(\widehat{r})+\beta Q(r(x),\widehat{r}{)}^p+\mathrm{\parallel }Px{\mathrm{\parallel }}_X^p,(\widehat{r},x)\in \tilde{U}\times U\mathrm{.}$

The joint source-condition objective takes $R_P=0$. The term $\beta Q(r(x),\widehat{r}{)}^p$ ties the relaxed range variable to the physical variable, while $\mathrm{\parallel }Px{\mathrm{\parallel }}_X^p$ penalizes the component outside $X_0$.

If $r$ is Gâteaux differentiable at a current iterate $x_n\in U$, the frozen-Newton objective with parameters ${\alpha }_n,{\beta }_n>0$ is

${\mathcal{J}}_n^{\delta }(\widehat{r},x):=\mathrm{\parallel }K\widehat{r}+F(x_0)-y^{\delta }{\mathrm{\parallel }}_Y^p+{\alpha }_n\mathcal{R}(\widehat{r})+{\beta }_{n}Q(r(x_n)+r^{\mathrm{\prime }}(x_n)(x-x_n),\widehat{r}{)}^p+\mathrm{\parallel }Px{\mathrm{\parallel }}_X^p\mathrm{.}$

Here $r^{\mathrm{\prime }}(x_n)$ is the bounded linear Gâteaux derivative used to form the affine range proxy; no minimizer existence or uniqueness is part of either definition.

The range factorization and its Banach-space maps

Proper convex regularizers on real Banach spaces

Barbara Kaltenbacher, Convergence rates under a range invariance condition with application to electrical impedance tomography, equations (1.8), (1.11), (2.7), (2.20), and (2.35)