Spectral regularization and parameter choice — Examples

Here $H_{\lambda }=1\mathrm{/}(1+\lambda )$, the residual is $(1-H_{\lambda })b=\lambda \mathrm{/}(1+\lambda )$, and $m=1$. Therefore the numerator and denominator of the score are both $(\lambda \mathrm{/}(1+\lambda ){)}^2$, so $GCV(\lambda )=1$ for each of the two positive grid values.

By [F3], the tied scores select the smaller parameter ${\lambda }_1=1\mathrm{/}10$. The corresponding estimate is $x_{1\mathrm{/}10}=10\mathrm{/}11$; at the other grid value it is $x_{10}=1\mathrm{/}11$.

Since the target is zero, the squared errors are $(10\mathrm{/}11{)}^2=100\mathrm{/}121$ and $(1\mathrm{/}11{)}^2=1\mathrm{/}121$. The first is one hundred times the second, so this valid finite GCV selection need not minimize solution error on its grid.

Relabelling coordinates by $k\mapsto k+1$ is an isometry from the complete space ${\mathrm{\ell }}^2(\mathbb{N},\mathbb{R})$ in [F2] to ${\mathrm{\ell }}^2({\mathbb{N}}_{+},\mathbb{R})$, so the latter is Hilbert. For $M\ge 1$, let $A_M{e}_n=n^{-1}{e}_n$ for $n\le M$ and $A_M{e}_n=0$ for $n>M$. Since $n^{-1}\le 1$, both $A$ and $A_M$ are bounded by one, and $A_M$ has finite rank. Also $\mathrm{\parallel }(A-A_M)x{\mathrm{\parallel }}_2^2={\sum }_{n>M}{n}^{-2}\mathrm{\mid }{x}_n{\mathrm{\mid }}^2\le (M+1{)}^{-2}\mathrm{\parallel }x{\mathrm{\parallel }}_2^2$, with equality at $x=e_{M+1}$. Thus $\mathrm{\parallel }A-A_M\mathrm{\parallel }=1\mathrm{/}(M+1)\to 0$. By [F3]–[F5], $A$ is a bounded compact operator on a Hilbert space.

For every $x\in {\mathrm{\ell }}^2$, the coordinate inner product gives $⟨Ax,e_n⟩=n^{-1}{x}_n=⟨x,n^{-1}{e}_n⟩$, so the adjoint identity yields $A^{\ast }{e}_n=n^{-1}{e}_n$ and $S{e}_n=n^{-2}{e}_n$. Thus the source target is $x^{†}=e_1+N^{-2\nu }{e}_N$. Its forward image is $y=e_1+N^{-(2\nu +1)}{e}_N$. The chosen noise cancels the $e_N$ datum, so $y_{\delta }=y+e=e_1$ and $\mathrm{\parallel }e\mathrm{\parallel }=\delta =N^{-(2\nu +1)}$.

The Tikhonov filter applied to $y_{\delta }=e_1$ gives $x_{\lambda }^{\delta }=(1+\lambda {)}^{-1}{e}_1$. Hence $x^{†}-x_{\lambda }^{\delta }=\frac{\lambda }{1+\lambda }{e}_1+N^{-2\nu }{e}_N,$ and, at $\lambda =N^{-2}$, $\mathrm{\parallel }{x}^{†}-x_{\lambda }^{\delta }{\mathrm{\parallel }}_2=\sqrt{(N^2+1{)}^{-2}+N^{-4\nu }}\mathrm{.}$ The exact-data bias in the $e_N$ coordinate is $\lambda N^{-2\nu }\mathrm{/}(N^{-2}+\lambda )=\frac{1}{2}{N}^{-2\nu }$; the noise output in that coordinate is $(1\mathrm{/}N)(N^{-2}+\lambda {)}^{-1}(-N^{-(2\nu +1)})=-\frac{1}{2}{N}^{-2\nu }$. Their contributions to solution error add.

For $0<\nu \le 1$, $N^{-2}\le N^{-2\nu }$, so the displayed error lies between $N^{-2\nu }$ and $\sqrt{2}{N}^{-2\nu }$. Also $\delta \mathrm{/}(2\sqrt{\lambda })=\frac{1}{2}{N}^{-2\nu }$ and ${\left(\frac{\delta }{\sqrt{2}}\right)}^{2\mathrm{/}(2\nu +1)}=N^{-2}{2}^{-1\mathrm{/}(2\nu +1)},$ so the selected $\lambda =N^{-2}$ differs from the balancing value by a factor between $1$ and $2$. These exact coordinates exhibit the stated bias-noise balance.

For the separate exponent $\mu >1$, take the unit source $w=e_1$. Since $S{e}_1=e_1$ by step 1.2, the source target is $x^{†}=S^{\mu }w=e_1$ and the exact datum is $y=A{e}_1=e_1$. For every $x$, $⟨Ax,e_1⟩=x_1=⟨x,e_1⟩$, so the adjoint identity gives $A^{\ast }{e}_1=e_1$. The Tikhonov solution is therefore $x_{\lambda }=(1+\lambda {)}^{-1}{e}_1$. Hence $\frac{\mathrm{\parallel }{x}^{†}-x_{\lambda }\mathrm{\parallel }}{{\lambda }^{\mu }}=\frac{{\lambda }^{1-\mu }}{1+\lambda }⟶+\mathrm{\infty }(\lambda \downarrow 0)\mathrm{.}$ This unit source therefore has bias not $O({\lambda }^{\mu })$.

Relabelling coordinates by $k\mapsto k+1$ is an isometry from the complete space ${\mathrm{\ell }}^2(\mathbb{N},\mathbb{R})$ in [F2] to ${\mathrm{\ell }}^2({\mathbb{N}}_{+},\mathbb{R})$, so the latter is Hilbert. Truncate $A$ after coordinate $M\ge 1$. Each diagonal multiplier has modulus at most one, so both $A$ and its finite-rank truncation are bounded by one; the operator-norm error is $1\mathrm{/}(M+1)$. By [F3]–[F4] and [F2], [F10], $A$ is compact on the Hilbert space [F1]. Its range is contained in $e_1^{\perp }$. Given $z\in e_1^{\perp }$ and $\epsilon >0$, choose a finite $F\subset \{2,3,\dots \}$ with ${\sum }_{n\mathrm{\notin }F}\mathrm{\mid }{z}_n{\mathrm{\mid }}^2<{\epsilon }^2$ using the finite-subsum definition in [F1]. The truncation $z_F={\sum }_{n\in F}{z}_n{e}_n$ lies in $\mathrm{Ran}A$, since $A({\sum }_{n\in F}n{z}_n{e}_n)=z_F$, and $\mathrm{\parallel }z-z_F\mathrm{\parallel }<\epsilon$. Thus $\overline{\mathrm{Ran}A}=e_1^{\perp }$.

The datum $y_{\delta }=2\delta e_1$ is orthogonal to $\overline{\mathrm{Ran}A}$, so $\mathrm{dist}(y_{\delta },\overline{\mathrm{Ran}A})=2\delta$ and $\mathrm{\parallel }{y}_{\delta }\mathrm{\parallel }=2\delta$. For every $x$, the first coordinate of $Ax$ is zero, so $⟨Ax,e_1⟩=0$. By [F8], $⟨x,A^{\ast }{e}_1⟩=0$ for all $x$; taking $x=A^{\ast }{e}_1$ and using [F9] gives $A^{\ast }{e}_1=0$. Thus [F5] gives $x_{\lambda }=0$ and the residual is exactly $2\delta$ for every $\lambda >0$.

The exact endpoint inputs are $d=G=2\delta$, while the declared target is $\tau \delta =3\delta \mathrm{/}2$. The compatibility condition in [F7] fails, and the target is below the residual floor from step 2.1. Thus no exact discrepancy crossing exists and the finite routine reports incompatible-endpoints, not a parameter.