Adaptive spectral regularization by SOLIT — Examples and Counterexamples

The functions $e_j(x)=\sqrt{2}\sin (\pi jx)$ are orthonormal on $(0,1)$ by the product-to-sum identities and direct integration. If $h\in L^2(0,1)$ is orthogonal to all of them, its odd extension to $(-1,1)$, continued periodically with period $2$, has zero constant and cosine coefficients by oddness and zero sine coefficients by the assumed orthogonality. Rescaling to the circle in [F1], every Fourier coefficient of this extension vanishes; completeness in [F2] therefore makes the extension, and hence $h$, zero almost everywhere. Thus $(e_j)$ is a complete orthonormal basis, and [F7] supplies the real Hilbert structure used below.

For $f\in L^2(0,1)$, [F8] gives $f\in L^1(0,1)$. Since $0\le G(x,t)\le 1\mathrm{/}4$, $\mathrm{\mid }Tf(x)\mathrm{\mid }\le \mathrm{\parallel }f{\mathrm{\parallel }}_1\mathrm{/}4\le \mathrm{\parallel }f{\mathrm{\parallel }}_2\mathrm{/}4$ for every $x$, so the integral formula defines a bounded linear operator from $L^2(0,1)$ to itself. The kernel formula also gives $(Tf)(x)=(1-x){\int }_0^{x}tf(t)dt+x{\int }_x^1(1-t)f(t)dt\mathrm{.}$ The two indefinite integrals are absolutely continuous by [F9], and [F10] gives their derivatives almost everywhere. They show that $Tf$ is continuous, has zero endpoint values, and has first derivative almost everywhere equal to the absolutely continuous function $g(x)=-{\int }_0^{x}tf(t)dt+{\int }_x^1(1-t)f(t)dt$, whose derivative is $g^{\mathrm{\prime }}(x)=-xf(x)-(1-x)f(x)=-f(x)$ almost everywhere. To verify the claimed distributional equation directly, take any $\varphi \in C_c^{\mathrm{\infty }}(0,1)$. The function $(x,t)\mapsto G(x,t)f(t){\varphi }^{\mathrm{\prime }\mathrm{\prime }}(x)$ is integrable on $(0,1{)}^2$, so [F11] permits reversing the integrals; splitting at $x=t$ and integrating the smooth test function twice gives ${\int }_0^1(Tf)(x){\varphi }^{\mathrm{\prime }\mathrm{\prime }}(x)dx={\int }_0^{1}f(t)({\int }_0^{1}G(x,t){\varphi }^{\mathrm{\prime }\mathrm{\prime }}(x)dx)dt=-{\int }_0^{1}f(t)\varphi (t)dt\mathrm{.}$ Thus $(Tf{)}^{\mathrm{\prime }\mathrm{\prime }}=-f$ in distributions. If two continuous Dirichlet solutions exist, their difference $h$ has $h^{\mathrm{\prime }\mathrm{\prime }}=0$ in distributions. Applying [F12] to $h^{\mathrm{\prime }}$ makes $h^{\mathrm{\prime }}$ constant, and applying it to $h-cx$ makes $h$ affine; continuity identifies the distribution with its pointwise affine representative. The two zero endpoint values force $h=0$.

For every $\lambda \ge 0$, the Tikhonov filter is nonnegative and satisfies $\alpha q_{\alpha }(\lambda )=\alpha \mathrm{/}(\lambda +\alpha )\le 1$ and $\lambda q_{\alpha }(\lambda )=\lambda \mathrm{/}(\lambda +\alpha )\le 1$. If $\tilde{\alpha }<\alpha$, then $\lambda +\tilde{\alpha }<\lambda +\alpha$, so $q_{\alpha }(\lambda )\le q_{\tilde{\alpha }}(\lambda )$. Thus the Tikhonov family is an ordered filter with $C_q^{\mathrm{\prime }}=1$.

Since $-e_j^{\mathrm{\prime }\mathrm{\prime }}=(\pi j{)}^2{e}_j$ and $e_j(0)=e_j(1)=0$, uniqueness of the Dirichlet solution from step 1.2 gives $T{e}_j=(\pi j{)}^{-2}{e}_j$. Completeness from step 1.1 therefore diagonalizes $T$ and $T^{\ast }T$, with singular values $s_j=(\pi j{)}^{-2}$ and eigenvalues ${\lambda }_j=(\pi j{)}^{-4}$. All singular values are positive, so $T$ is injective. The truncations $T_{N}f={\sum }_{j=1}^N(\pi j{)}^{-2}⟨f,e_j⟩e_j$ have finite rank and satisfy $\mathrm{\parallel }T-T_N\mathrm{\parallel }={\sup }_{j>N}(\pi j{)}^{-2}\to 0$, so $T$ is compact. Also ${\sum }_j{s}_j^2={\pi }^{-4}{\sum }_j{j}^{-4}<\mathrm{\infty }$ by comparison with ${\int }_1^{\mathrm{\infty }}{x}^{-4}dx$, so $T$ is Hilbert–Schmidt.

For the Tikhonov filter, put $V(\alpha ):={\sum }_{j\ge 1}{\lambda }_j({\lambda }_j+\alpha {)}^{-2}$. On every compact interval $[a,b]\subset (0,\mathrm{\infty })$ its summands are continuous and bounded above by ${\lambda }_j\mathrm{/}{a}^2$, a summable sequence by step 2.1, so $V$ is continuous there. Each summand strictly decreases with $\alpha$. Also $V(\alpha )\le {\alpha }^{-2}{\sum }_j{\lambda }_j\to 0$ as $\alpha \to \mathrm{\infty }$. For every fixed $j$, $V(\alpha )\ge {\lambda }_j({\lambda }_j+\alpha {)}^{-2}\to 1\mathrm{/}{\lambda }_j$ as $\alpha \downarrow 0$; since ${\lambda }_j\to 0$, these lower bounds show $V(\alpha )\to \mathrm{\infty }$. Given $c>0$ and $\theta >1$, continuity and strict decrease give a unique ${\alpha }_m>0$ with $V({\alpha }_m)=c{\theta }^m$ for each $m\in {\mathbb{N}}_0$. Thus ${\alpha }_m$ decreases to zero and $v_m={\sigma }^{2}V({\alpha }_m)$ has consecutive ratio $\theta$, so [F4] holds.

Direct integration of the two pieces of $f^{†}$ gives $⟨f^{†},e_j⟩=2\sqrt{2}\sin (\pi j\mathrm{/}2)\mathrm{/}({\pi }^2{j}^2)$. On the first half, $g^{†}(x)=-x^3\mathrm{/}6+x\mathrm{/}8$, so $g^{†\mathrm{\prime }\mathrm{\prime }}(x)=-x$; on the second half, $g^{†}(x)=x^3\mathrm{/}6-x^2\mathrm{/}2+3x\mathrm{/}8-1\mathrm{/}24$, so $g^{†\mathrm{\prime }\mathrm{\prime }}(x)=x-1$. The two values at $x=1\mathrm{/}2$ both equal $1\mathrm{/}24$, and both first derivatives there equal zero; the endpoint values are zero. Hence $-g^{†\mathrm{\prime }\mathrm{\prime }}=f^{†}$ weakly with Dirichlet boundary values, so uniqueness from step 1.2 gives $T{f}^{†}=g^{†}$. The sequence model in [F3] and the filter conditions in [F4] therefore supply an admissible white-noise input to the finite selector in [F5], whose estimates are well-defined by [F6].

Interleaving $(a_n,b_n{)}_{n\ge 1}$ as $(a_1,b_1,a_2,b_2,\dots )$ is a linear isometry onto real ${\mathrm{\ell }}^2({\mathbb{N}}_{>0})$, so [F7] gives the stated real Hilbert structure. In angular coordinates with normalized Haar measure $dx\mathrm{/}(2\pi )$, write the characters as $e_n(x)=e^{inx}$ and use $C_n=(e_n+e_{-n})\mathrm{/}\sqrt{2}=\sqrt{2}\cos (nx)$ and $S_n=(e_n-e_{-n})\mathrm{/}(i\sqrt{2})=\sqrt{2}\sin (nx)$ for $n\ge 1$. Their orthonormality follows from character orthonormality in [F1]. They are complete in mean-zero real $L^2$: a real mean-zero function orthogonal to all $C_n,S_n$ is orthogonal to every character, hence is zero by [F1]. Thus their coefficient map identifies this function space isometrically with $X$. The bounded truth belongs to $L^2$, and its integral over $[-\pi ,\pi ]$ is ${\pi }^2-{\pi }^2=0$, so its constant Fourier coefficient vanishes and its normalized real Fourier coefficient sequence lies in $X$.

For every $n\ge 1$, $t_n>0$ and $t_n\to 0$ exponentially since $R>1$. The ratio of consecutive squared multipliers is $((n+1)\mathrm{/}n{)}^2((n+2)\mathrm{/}(n+1){)}^2{R}^{-2}$, which tends to $R^{-2}<1$; choose $\rho$ strictly between $R^{-2}$ and $1$, then the ratio is at most $\rho$ for all sufficiently large $n$. Thus ${\sum }_{n\ge 1}2t_n^2<\mathrm{\infty }$. The operator is injective because all its diagonal multipliers are positive. Its truncation to frequencies $n\le N$ is finite rank and the operator-norm error is ${\sup }_{n>N}{t}_n\to 0$, so $T_R$ is compact and Hilbert–Schmidt.

The frequency multipliers satisfy $t_{n+1}\mathrm{/}{t}_n=(n+2)\mathrm{/}(nR)$, which can exceed $1$ for small $n$; thus the frequency order is not always the singular-value order. The multiset of modes has a nonincreasing enumeration: at every stage a remaining maximum exists because $t_n\to 0$ and only finitely many modes lie above any fixed positive threshold, and the fixed tie rule selects one uniquely. Every mode is eventually listed because only finitely many modes have multiplier at least its positive multiplier. The operator is diagonal on the orthonormal cosine–sine basis, so the frequency eigenvalue is ${\mu }_n=t_n^2=n^2(n+1{)}^2{R}^{-2n-4}$ twice at each frequency, and the enumerated singular system has these values in nonincreasing order. Since $(n+1{)}^2\asymp n^2$, ${\mu }_n\asymp n^4{e}^{-2n\log R}$, an exponential spectrum with exponent $\vartheta =1$.

For every $\lambda \ge 0$, $q_{\alpha }(\lambda )=(\lambda +\alpha {)}^{-1}$ is nonnegative, $\alpha q_{\alpha }(\lambda )\le 1$, and $\lambda q_{\alpha }(\lambda )\le 1$. If $\tilde{\alpha }<\alpha$, then $\lambda +\tilde{\alpha }<\lambda +\alpha$, so $q_{\alpha }(\lambda )\le q_{\tilde{\alpha }}(\lambda )$. Thus it is an ordered filter with $C_q^{\mathrm{\prime }}=1$ by [F6]. Its trace variance is

On every compact interval $[a,b]\subset (0,\mathrm{\infty })$, each summand is continuous and is bounded above by $2{\mu }_n\mathrm{/}{a}^2$. Since ${\sum }_{n}2{\mu }_n<\mathrm{\infty }$ by step 1.2, the series converges uniformly there, so $V$ is continuous. Each summand strictly decreases as $\alpha$ increases, hence so does $V$. Also $V(\alpha )\le 2{\alpha }^{-2}{\sum }_n{\mu }_n\to 0$ as $\alpha \to \mathrm{\infty }$. For every fixed $n$, $V(\alpha )\ge 2{\mu }_n\mathrm{/}({\mu }_n+\alpha {)}^2\to 2\mathrm{/}{\mu }_n$ as $\alpha \downarrow 0$; because ${\mu }_n\to 0$, these lower bounds imply $V(\alpha )\to \mathrm{\infty }$. Fix any $c>0$ and $\theta >1$. Continuity, strict decrease, and these endpoint limits give a unique ${\alpha }_m>0$ with $V({\alpha }_m)=c{\theta }^m$ for every $m\in {\mathbb{N}}_0$. Thus ${\alpha }_m$ decreases to zero and the variances $v_m={\sigma }^{2}V({\alpha }_m)$ have the constant consecutive ratio $\theta$. The resulting infinite grid meets [F2], and the SOLIT rule truncates it at its finite $m_{\max }$.

The truth lies in $L^2$ because it is bounded, and its Fourier coefficients are coordinates in the model by [F1]. The positive diagonal multipliers give the injective compact Hilbert–Schmidt operator verified in step 1.2; the Tikhonov filter satisfies [F6] and its candidates satisfy [F2], and the positive-noise experiment and finite selector are [F3]–[F4]. The well-definedness and moment properties of its estimators follow from [F5]. This example asserts no rate for the resulting estimator.

Interleaving the constant coordinate and cosine–sine pairs is a linear isometry onto real ${\mathrm{\ell }}^2({\mathbb{N}}_{>0})$, so [F7] gives the stated real Hilbert structure. With normalized Haar measure $dx\mathrm{/}(2\pi )$ and characters $e_n(x)=e^{inx}$, use the real modes $1$, $C_n=(e_n+e_{-n})\mathrm{/}\sqrt{2}=\sqrt{2}\cos (nx)$ and $S_n=(e_n-e_{-n})\mathrm{/}(i\sqrt{2})=\sqrt{2}\sin (nx)$ for $n\ge 1$. Character orthonormality in [F1] gives their orthonormality. They are complete in real $L^2$: a real function orthogonal to all these modes is orthogonal to every character, hence is zero by [F1]. Thus their coefficient map is an isometry onto the stated coefficient space. The bounded truth lies in $L^2$ and has such coordinates. On each positive frequency, ${\mathrm{\partial }}_{xx}{C}_n=-n^2{C}_n$ and ${\mathrm{\partial }}_{xx}{S}_n=-n^2{S}_n$, so solving ${\mathrm{\partial }}_{t}u={\mathrm{\partial }}_{xx}u$ multiplies both coefficients by $e^{-n^2\bar{t}}$; the constant mode is unchanged. Therefore the operator in the Example is the periodic heat forward map in normalized Fourier coordinates.

Every multiplier of $T_{\bar{t}}$ is positive, so $T_{\bar{t}}$ is injective. The positive-frequency multipliers tend to zero, so finite-frequency truncations converge in operator norm and $T_{\bar{t}}$ is compact. Moreover, ${\sum }_{n\ge 1}2e^{-2n^2\bar{t}}<\mathrm{\infty }$ by comparison with the geometric series ${\sum }_{n\ge 1}2e^{-2n\bar{t}}$, so the operator is Hilbert–Schmidt. Squaring the multipliers gives the stated eigenvalues of $T_{\bar{t}}^{\ast }{T}_{\bar{t}}$.

For spectral cut-off, the inverse-eigenvalue contribution at frequency $n$ is $2e^{2n^2\bar{t}}$. Since $V_{n-1}\le (2n-1)e^{2(n-1{)}^2\bar{t}}$,

Hence no finite ${\theta }_2$ bounds all consecutive ratios in this sequence, as claimed.

For every $\lambda \ge 0$, $q_{\alpha }(\lambda )=(\lambda +\alpha {)}^{-1}$ is nonnegative, $\alpha q_{\alpha }(\lambda )\le 1$, and $\lambda q_{\alpha }(\lambda )\le 1$. If $\tilde{\alpha }<\alpha$, then $\lambda +\tilde{\alpha }<\lambda +\alpha$, so $q_{\alpha }(\lambda )\le q_{\tilde{\alpha }}(\lambda )$. Thus it is an ordered filter with $C_q^{\mathrm{\prime }}=1$ by [F6]. Its variance is $V(\alpha )=1\mathrm{/}(1+\alpha {)}^2+2{\sum }_{n\ge 1}{e}^{-2n^2\bar{t}}(e^{-2n^2\bar{t}}+\alpha {)}^{-2}$. On each compact interval $\alpha \ge a>0$, the summands are continuous and bounded by $2a^{-2}{e}^{-2n^2\bar{t}}$, a summable sequence, so $V$ is continuous there by uniform convergence. It is strictly decreasing. For $\alpha \ge 1$, $V(\alpha )\le {\alpha }^{-2}(1+2{\sum }_{n\ge 1}{e}^{-2n^2\bar{t}})\to 0$ as $\alpha \to \mathrm{\infty }$. It tends to infinity as $\alpha \downarrow 0$: for every $N$, its first $N$ positive-frequency terms tend to $2{\sum }_{n=1}^N{e}^{2n^2\bar{t}}$, and these partial sums are unbounded in $N$. The intermediate value theorem therefore selects recursively a decreasing sequence with any prescribed fixed ratio ${\theta }_1>1$ between consecutive variance values; its upper ratio is the same ${\theta }_1$. The estimator and each finite comparison are well-defined by [F3]–[F5].