Gaussian sequence inverse model

Assume the Axiom of Countable Choice. Let $X$ and $Y$ be real Hilbert spaces and let $K:X\to Y$ be a compact linear operator of infinite rank, viewed as an inverse model with exact datum $Kx$. Here compact means that the image of every bounded subset has compact closure. Write $N(K)=\{x\in X:Kx=0\}$ and $H_0=\overline{\mathrm{ran}K}$. Fix a positive singular system $(s_j,v_j,u_j{)}_{j\in {\mathbb{N}}_{>0}}$ for $K$, where ${\mathbb{N}}_{>0}=\{1,2,3,\dots \}$, $s_1\ge s_2\ge \cdots >0$, $s_j\to 0$, $K{v}_j=s_j{u}_j$, $(v_j)$ is a complete orthonormal basis of $N(K{)}^{\perp }$, and $(u_j)$ is a complete orthonormal basis of $H_0$. A complete orthonormal basis is an orthonormal family whose closed linear span is the whole indicated subspace. These are the compact SVD conventions; in particular, this model concerns minimum-norm representatives in $N(K{)}^{\perp }$ and does not estimate a kernel component.

For a truth $x\in N(K{)}^{\perp }$, let ${\theta }_j=⟨x,v_j{⟩}_X$. The complete-basis coefficient map identifies $N(K{)}^{\perp }$ bijectively and isometrically with ${\mathrm{\ell }}^2({\mathbb{N}}_{>0},\mathbb{R})$, so every $\theta \in {\mathrm{\ell }}^2({\mathbb{N}}_{>0},\mathbb{R})$ is a model parameter. For $ϵ>0$, take the functional white-noise experiment on the separable Hilbert space $H_0$ with observations $Y_x(g)=⟨Kx,g{⟩}_Y+ϵW(g)$, where $W$ is centered, has covariance $\mathbb{E}[W(g)W(h)]=⟨g,h{⟩}_Y$, and has independent standard-normal values on every finite orthonormal tuple. The countable complete basis makes $H_0$ separable. Define ${\xi }_j=W(u_j)$ and $Z_j=Y_x(u_j)\mathrm{/}{s}_j$. The sequence sample space is ${\mathbb{R}}^{{\mathbb{N}}_{>0}}$ with the product Borel sigma-algebra, meaning the sigma-algebra generated by its coordinate projections. The coordinate law is the pushforward of the functional experiment under $y\mapsto (y(u_j)\mathrm{/}{s}_j{)}_{j\in {\mathbb{N}}_{>0}}$; this map is measurable because each coordinate is a cylinder evaluation followed by scalar multiplication. The compact SVD expansion gives $⟨Kx,u_j{⟩}_Y=s_j{\theta }_j$, and the positive-noise white-noise coordinate theorem gives the independent standard-normal coordinates ${\xi }_j$. Hence the normalized coordinates satisfy $Z_j={\theta }_j+ϵs_j^{-1}{\xi }_j,j\in {\mathbb{N}}_{>0}\mathrm{.}$

For the zero-noise boundary $ϵ=0$, define the sequence experiment separately by the point mass at $\theta$, so $Z_j={\theta }_j$. This boundary is not an instance of the positive-noise functional experiment.

Compact linear inverse model between Hilbert spaces

Compact-operator singular system and expansion

Complete orthonormal systems and Hilbert bases

A complete orthonormal basis identifies a Hilbert space isometrically with ${\mathrm{\ell }}^2$

Square-summable families and ${\mathrm{\ell }}^2$

Functional isonormal white-noise experiment

Independent Gaussian coordinates in a white-noise experiment

The Axiom of Countable Choice

Litao Ding and Peter Mathé, Minimax rates for statistical inverse problems under general source conditions, §1, equations (1)–(4), PDF pp. 1–2

Iain M. Johnstone, Gaussian estimation: Sequence and wavelet models, Chapter 3, equations (3.63)–(3.67), printed p. 78; §3.8 begins on p. 77

The deterministic square commutes: ${\mathrm{\Phi }}_X(x)=(⟨x,v_j⟩{)}_j$ and ${\mathrm{\Phi }}_{H_0}(y)=(⟨y,u_j⟩{)}_j$ are the coefficient isometries, and $D_s(\theta )=(s_j{\theta }_j{)}_j$ is well defined because $s_j\le s_1$. The compact SVD expansion gives ${\mathrm{\Phi }}_{H_0}(Kx)=D_s({\mathrm{\Phi }}_X(x))$. The noisy coordinates add $ϵ{\xi }_j$ before division by $s_j$; the diagram is not a claim that white noise is an $H_0$-valued sample and is not used as a proof.