Linear feature identifiability and Gaussian MAP tests

Fix a probe $\mathrm{\Phi }\in H$. By [F1] and [F2], whose supplier hypotheses are recorded in [A1], its mean at parameter $x$ is $⟨x,K^{\ast }\mathrm{\Phi }{⟩}_X$ and its variance is finite, equal to ${ϵ}^2\mathrm{\parallel }\mathrm{\Phi }{\mathrm{\parallel }}_H^2$. If it is unbiased for every $x$, then $⟨x,K^{\ast }\mathrm{\Phi }-h_{\varphi }{⟩}_X=0$ for every $x\in X$. Taking $x=K^{\ast }\mathrm{\Phi }-h_{\varphi }$ gives $K^{\ast }\mathrm{\Phi }=h_{\varphi }$, so $h_{\varphi }\in R(K^{\ast })$.

Conversely, if $h_{\varphi }\in R(K^{\ast })$, use [A1] to choose $\mathrm{\Phi }\in H$ with $K^{\ast }\mathrm{\Phi }=h_{\varphi }$. Equations [F1] and [F2] then give ${\mathbb{E}}_x{Y}_x(\mathrm{\Phi })=⟨x,h_{\varphi }{⟩}_X=\varphi (x)$ for every $x$, and [F1] gives its finite variance ${ϵ}^2\mathrm{\parallel }\mathrm{\Phi }{\mathrm{\parallel }}_H^2$. Hence this probe is an exact one-probe estimator, proving both directions and the variance assertion.

Set $S=C_0^{1\mathrm{/}2}$. The positive trace-class operator $C_0$ is compact by [F4], so $S$ is compact and self-adjoint. Then $B=S{K}^{\ast }KS$ is compact, self-adjoint, and positive, because a composition containing a compact operator is compact. By [F5], applied to $KS$, $B+{ϵ}^2{I}_X$ is bijective and $R$ is bounded. Its inverse is self-adjoint: writing $x=(B+{ϵ}^{2}I)u$ and $z=(B+{ϵ}^{2}I)v$ transfers symmetry of $B+{ϵ}^{2}I$ to $R$. It is positive because $⟨Rz,z⟩=⟨v,(B+{ϵ}^{2}I)v⟩\ge {ϵ}^2\mathrm{\parallel }v{\mathrm{\parallel }}^2$ when $z=(B+{ϵ}^{2}I)v$. The inverse identities give $BR=RB=I-{ϵ}^{2}R$. For $z\in X$, put $v=Rz$, so $z=(B+{ϵ}^{2}I)v$. Expanding this equation and using self-adjointness of $B$ gives $\mathrm{\parallel }z{\mathrm{\parallel }}^2-{ϵ}^2⟨Rz,z⟩=\mathrm{\parallel }Bv{\mathrm{\parallel }}^2+{ϵ}^2⟨Bv,v⟩\ge 0$. Thus $0\le {ϵ}^{2}R\le I_X$, so $BR=I-{ϵ}^{2}R$ is positive and $I-BR={ϵ}^{2}R\ge 0$. Define $D=SBRS$; then $D$ is positive, $C_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}}={ϵ}^{2}SRS$ is positive, and $D+C_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}}=S(BR+{ϵ}^{2}R)S=C_0$. The products are trace class by [F8] because $S$ is Hilbert–Schmidt and the intervening operators are bounded. Consequently $0\le D\le C_0$ and $\mathrm{tr}D\le \mathrm{tr}{C}_0$.

For $g\in H$, write $Y_0(g)=Y(g)-⟨K{m}_0,g{⟩}_H$. The signal and noise are independent, so covariance expansion gives $\mathbb{E}[Y_0(g)Y_0(k)]=⟨K{C}_0{K}^{\ast }g,k{⟩}_H+{ϵ}^2⟨g,k{⟩}_H$. For $h\in X$, $Y_0({\mathrm{\Phi }}_h)=y({\mathrm{\Phi }}_h)-⟨K{m}_0,{\mathrm{\Phi }}_h{⟩}_H$ when the observed field is $y$. Using $S{K}^{\ast }{\mathrm{\Phi }}_h=BRSh$ and ${\mathrm{\Phi }}_h=KSRSh$, the covariance identity yields $\mathbb{E}[Y_0({\mathrm{\Phi }}_h)Y_0({\mathrm{\Phi }}_k)]=⟨SBRSh,k{⟩}_X=⟨Dh,k{⟩}_X$. Likewise $\mathbb{E}[⟨U-m_0,h{⟩}_X{Y}_0({\mathrm{\Phi }}_k)]=⟨Dh,k{⟩}_X$. These equalities follow by expanding the two covariance terms, using $BR=RB$ and $(B+{ϵ}^{2}I)R=I$.

For the supplied finite or countable basis $(v_j)$ of $X$, [F8] and step 1.1 give $\mathbb{E}{\sum }_j\mathrm{\mid }{Y}_0({\mathrm{\Phi }}_{v_j}){\mathrm{\mid }}^2={\sum }_j⟨D{v}_j,v_j{⟩}_X=\mathrm{tr}D\le \mathrm{tr}{C}_0<\mathrm{\infty }$. Therefore the nonnegative sum is finite almost surely: if it were infinite on a set of positive probability, its expectation would exceed every finite bound. Parseval gives ${\sum }_j\mathrm{\mid }⟨m_0,v_j{⟩}_X{\mathrm{\mid }}^2=\mathrm{\parallel }{m}_0{\mathrm{\parallel }}_X^2<\mathrm{\infty }$, and $\mathrm{\mid }a+b{\mathrm{\mid }}^2\le 2\mathrm{\mid }a{\mathrm{\mid }}^2+2\mathrm{\mid }b{\mathrm{\mid }}^2$ shows ${\sum }_j{b}_j(Y{)}^2<\mathrm{\infty }$ on that probability-one set. Hence the first branch of the Statement's definition applies and $M_Y=m_0+{\sum }_j{Y}_0({\mathrm{\Phi }}_{v_j})v_j$ there. The same argument includes finite and empty bases. The series converges in $X$ by [F7]. Its partial sums are measurable; their Cauchy set is a countable intersection and union of norm-comparison events, hence measurable, and completeness of $X$ gives a limit there. For each nonempty closed set $F\subseteq X$, distance to $F$ is continuous, so the distance of the limit to $F$ is the limit of the measurable distances of the partial sums; [F11] makes this limit measurable. Because each coordinate map $y\mapsto y({\mathrm{\Phi }}_{v_j})$ is measurable for the cylinder sigma-algebra, the convergence construction and the default value $m_0$ show that $y\mapsto M_y$ is measurable from $({\mathbb{R}}^H,{\mathcal{C}}_H)$ to $X$ with its Borel sigma-algebra. For each fixed $h$, expand $h$ in the supplied basis. The corresponding finite linear combinations of $Y_0({\mathrm{\Phi }}_{v_j})$ converge in $L^2$ to $Y_0({\mathrm{\Phi }}_h)$, since [F7] makes the basis projections converge to $h$ and [step 2.1] bounds the squared error by $\mathrm{\parallel }D\mathrm{\parallel }$ times the squared projection error. The same finite combinations converge almost surely to the inner product of the coordinate-series limit with $h$. Uniqueness of limits in probability gives $⟨M_Y-m_0,h{⟩}_X=Y_0({\mathrm{\Phi }}_h)$ almost surely for this fixed $h$. This proves the displayed posterior-mean formula, with an exceptional set allowed to depend on $h$.

For fixed $h\in X$ and $g\in H$, covariance expansion gives $\mathrm{Cov}(⟨U-m_0,h{⟩}_X,Y_0(g))=⟨K{C}_{0}h,g{⟩}_H$. The covariance of $Y_0({\mathrm{\Phi }}_h)$ with $Y_0(g)$ is $⟨K{C}_0{K}^{\ast }{\mathrm{\Phi }}_h+{ϵ}^2{\mathrm{\Phi }}_h,g{⟩}_H$. Since $C_0=S^2$, $S{K}^{\ast }KS=B$, and $(B+{ϵ}^{2}I)R=I$, one has $K{C}_0{K}^{\ast }{\mathrm{\Phi }}_h+{ϵ}^2{\mathrm{\Phi }}_h=KSBRSh+{ϵ}^{2}KSRSh=KS(B+{ϵ}^{2}I)RSh=KSSh=K{C}_{0}h$. By step 3.1, $⟨M_Y-m_0,h⟩=Y_0({\mathrm{\Phi }}_h)$ almost surely, so the residual $W=U-M_Y$ has zero covariance with every observation $Y(g)$. Both $⟨U-m_0,h⟩$ and $Y_0({\mathrm{\Phi }}_h)$ are centered, so the same identity shows $\mathbb{E}⟨W,h⟩=0$ for each $h$; thus $W$ is centered. Its covariance is $C_0-D=C_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}}$ by the two covariance identities in step 2.1 and the covariance of $U$.

For each fixed basis vector $v_j$, step 3.1 identifies $⟨M_Y-m_0,v_j⟩$ almost surely with $Y_0({\mathrm{\Phi }}_{v_j})$. For any finite list of basis vectors, intersect the corresponding probability-one events; the associated coordinates of $W$ are then linear images of finite Gaussian projections of $U-m_0$ and $Z$. The same holds jointly with any finite list of observations. The underlying finite $U-m_0$ and $Z$ blocks are independent by [F12], so their joint linear images are Gaussian by [F3] and [F6]. Step 4.1 makes the residual-coordinate and observation blocks have zero cross-covariance, so [F6] makes those blocks independent. The rational finite linear combinations of the supplied basis $(v_j)$ form a countable dense subset of $X$. For each center in this set, Parseval expresses the squared distance from $W$ as the limit of finite sums of squared basis-coordinate differences, so open balls with rational radii about these centers are measurable in the basis-coordinate sigma-algebra; they form a countable base, proving that these coordinates generate the Borel sigma-algebra of $W$. For a fixed finite list of observations, finite basis-coordinate cylinder rectangles and observation rectangles have the product of their marginal probabilities by the independence just proved. These rectangles form a generating pi-system for the product of the residual Borel sigma-algebra and the finite observation Borel sigma-algebra, so [F9] extends the product identity and shows that $W$ is independent of this finite observation block. Now fix any $A\in \mathcal{B}(X)$. The finite measures $C\mapsto \mathbb{P}(W\in A,C)$ and $C\mapsto \mathbb{P}(W\in A)\mathbb{P}(C)$ agree on the pi-system of finite observation cylinder events, including the whole space. By [F9] they agree on $\sigma (Y(g):g\in H)$. Since this holds for every $A$, $W$ is independent of the full field. This argument includes finite and empty supplied bases and requires no inclusion in the ordinary completion of a basis-observation sigma-algebra.

The map $y\mapsto M_y$ is measurable by step 3.1. Let $\nu$ be the Borel law of the residual random element $W$, whose mean is zero and covariance is $C_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}}$ by step 4.1. Define ${\mathrm{\Pi }}_y(A)={\int }_X{\mathbf{1}}_A(M_y+w)\nu (dw)$ for $A\in \mathcal{B}(X)$. The map $(y,w)\mapsto M_y+w$ is measurable, and [F10] makes $y\mapsto {\mathrm{\Pi }}_y(A)$ measurable; for each $y$, translation of $\nu$ is a probability measure. Since $W$ is independent of the full data field by step 5.1 and $U=M_Y+W$, integrating first with respect to the fixed law $\nu$ gives $\mathbb{P}(U\in A,Y\in C)={\int }_C{\mathrm{\Pi }}_y(A)P_Y(dy)$ for every Borel $A\subseteq X$ and $C\in {\mathcal{C}}_H$. This is the regular conditional distribution identity from [F2].

For every finite list $h_1,\dots ,h_q\in X$, step 3.1 gives almost surely, simultaneously for this finite list, $⟨W,h_a⟩=⟨U-m_0,h_a⟩-Y_0({\mathrm{\Phi }}_{h_a})$. The vector of these residual functionals is therefore a linear image of finite Gaussian projections of $U-m_0$ and $Z$, whose blocks are independent by [F12] and jointly Gaussian by [F3] and [F6]; hence it is jointly Gaussian. Step 4.1 gives that $W$ is centered with covariance $C_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}}$, so $W$ is a centered Borel Gaussian random element with that covariance. Translating by $M_y$ shows that every section ${\mathrm{\Pi }}_y$ is Gaussian with mean $M_y$ and covariance $C_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}}$, including the zero-dimensional and finite-dimensional cases. For fixed $h$, step 3.1 identifies the posterior mean with $⟨m_0,h⟩+y({\mathrm{\Phi }}_h)-⟨K{m}_0,{\mathrm{\Phi }}_h⟩$ almost surely, and the scalar marginal under a translated Gaussian is normal with variance $⟨C_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}}h,h⟩$. This proves all asserted posterior-law and feature-marginal formulas; the exceptional set remains feature-dependent.

For fixed $x$ and $\mathrm{\Phi }\mathrm{\ne }0$, [F2] gives $Y_x(\mathrm{\Phi })=⟨Kx,\mathrm{\Phi }{⟩}_H+ϵ\mathrm{\parallel }\mathrm{\Phi }{\mathrm{\parallel }}_H{Z}_0$ in law, where $Z_0$ is standard normal. By [F7], the mean is $⟨x,K^{\ast }\mathrm{\Phi }{⟩}_X$. Hence ${\mathbb{P}}_x\{Y_x(\mathrm{\Phi })>c\}=1-G((c-⟨x,K^{\ast }\mathrm{\Phi }⟩)\mathrm{/}(ϵ\mathrm{\parallel }\mathrm{\Phi }\mathrm{\parallel }))$, which equals the stated formula by symmetry in [F3]. For $\mathrm{\Phi }=0$, $Y_x(0)=0$ and the probability is exactly $\mathbf{1}\{0>c\}$. Substituting ${\mathrm{\Phi }}_h,c_h$ proves the raw MAP formula.

For the weighted clause, [F4] gives the compact positive injective spectral decomposition $C_0{v}_j=c_j{v}_j$ with $c_j>0$, finite-dimensional eigenspaces, and a complete orthonormal family spanning $X$. The commutation hypothesis makes each such eigenspace invariant under the self-adjoint positive operator $K^{\ast }K$. By [F6], diagonalize its finite-dimensional restriction in each eigenspace. This gives a common complete orthonormal basis with $C_0{v}_j=c_j{v}_j$ and $K^{\ast }K{v}_j={\lambda }_j{v}_j$, ${\lambda }_j\ge 0$. For ${\lambda }_j>0$, put $w_j=K{v}_j\mathrm{/}\sqrt{{\lambda }_j}$; then $⟨w_i,w_j{⟩}_H={\delta }_{ij}$ by [F7]. Define $T{v}_j=\sqrt{{\lambda }_j}{v}_j$ and define $J{v}_j=w_j$ for ${\lambda }_j>0$ and $J{v}_j=0$ for ${\lambda }_j=0$. For a finite sum $x={\sum }_j{a}_j{v}_j$, orthonormality gives $\mathrm{\parallel }Tx{\mathrm{\parallel }}^2={\sum }_j{\lambda }_j\mathrm{\mid }{a}_j{\mathrm{\mid }}^2\le \mathrm{\parallel }K{\mathrm{\parallel }}^2\mathrm{\parallel }x{\mathrm{\parallel }}^2$ and $\mathrm{\parallel }Jx{\mathrm{\parallel }}^2={\sum }_{{\lambda }_j>0}\mathrm{\mid }{a}_j{\mathrm{\mid }}^2\le \mathrm{\parallel }x{\mathrm{\parallel }}^2$. Thus both rules extend by completeness to bounded real-linear operators on $X$, with $T$ self-adjoint and positive and $T^2=K^{\ast }K$. Any bounded self-adjoint positive square root $Q$ of $K^{\ast }K$ commutes with its square and therefore preserves each eigenspace; on an eigenspace with eigenvalue $\lambda \ge 0$, it equals $\sqrt{\lambda }I$. Indeed, for $Q^2=\lambda I$, $a=\sqrt{\lambda }$, and a vector $x$ in that eigenspace, $V=(Q-aI)x$ satisfies $QV=-aV$, which positivity forces to vanish; when $\lambda =0$, $\mathrm{\parallel }Qx{\mathrm{\parallel }}^2=⟨Q^{2}x,x⟩=0$. Hence $T=(K^{\ast }K{)}^{1\mathrm{/}2}$. Also $\mathrm{\parallel }Kx{\mathrm{\parallel }}^2={\sum }_j{\lambda }_j\mathrm{\mid }⟨x,v_j⟩{\mathrm{\mid }}^2$, so $\ker K$ is the closed span of the zero-eigenvalue vectors and $(\ker K{)}^{\perp }$ is the closed span of the positive-eigenvalue vectors. The extended $J$ vanishes on $\ker K$, is isometric on $(\ker K{)}^{\perp }$, and has $\ker J=\ker K$ by its coordinate formula; these real-space properties verify that $J$ is a partial isometry as characterized in [F5], including when the basis is empty. The expansions of $Kx$ and $Jx$ show $K=JT$ and $\overline{R(K)}=\stackrel{}{\mathrm{span}}\{w_j:{\lambda }_j>0\}$. Since $w_j$ are orthonormal, with ${\xi }_j=⟨\mathrm{\Psi },w_j{⟩}_H$ the adjoint identities and Fourier expansion in $X$ give $J^{\ast }\mathrm{\Psi }={\sum }_{{\lambda }_j>0}{\xi }_j{v}_j$ and $K^{\ast }\mathrm{\Psi }={\sum }_{{\lambda }_j>0}\sqrt{{\lambda }_j}{\xi }_j{v}_j$. If $h_j=⟨h,v_j{⟩}_X$, the Fourier expansions give $\mathrm{\parallel }{K}^{\ast }\mathrm{\Psi }-h{\mathrm{\parallel }}_X^2={\sum }_{{\lambda }_j>0}\mathrm{\mid }\sqrt{{\lambda }_j}{\xi }_j-h_j{\mathrm{\mid }}^2+{\sum }_{{\lambda }_j=0}\mathrm{\mid }{h}_j{\mathrm{\mid }}^2$ and $\mathrm{\parallel }{C}_0^{-1\mathrm{/}2}{J}^{\ast }\mathrm{\Psi }{\mathrm{\parallel }}_X^2={\sum }_{{\lambda }_j>0}\mathrm{\mid }{\xi }_j{\mathrm{\mid }}^2\mathrm{/}{c}_j$ whenever the second term is defined. Components of $\mathrm{\Psi }$ in ${\overline{R(K)}}^{\perp }$ affect neither expression.

For each fixed $h$, [F9] gives for $P_Y$-almost every $y$ that $⟨M_y,h{⟩}_X=⟨m_0,h{⟩}_X+y({\mathrm{\Phi }}_h)-⟨K{m}_0,{\mathrm{\Phi }}_h{⟩}_H=y({\mathrm{\Phi }}_h)-c_h$. For every $y$, the posterior feature marginal has the normal law stated in the theorem. If its variance ${\sigma }^2$ is positive, its density is $(\sigma \sqrt{2\pi }{)}^{-1}\exp (-(t-⟨M_y,h{⟩}_X{)}^2\mathrm{/}(2{\sigma }^2))$, uniquely maximized at $⟨M_y,h{⟩}_X$; if its variance is zero, it is the point mass at that same value. Thus its mode is $⟨M_y,h{⟩}_X$ in both cases. On the probability-one set where the displayed identity holds, the raw comparison $y({\mathrm{\Phi }}_h)>c_h$ is therefore equivalent to a positive posterior marginal mode; its exceptional null set may depend on $h$. If $h=0$, the definitions give ${\mathrm{\Phi }}_h=c_h=0$, and [F2] gives $Y_x(0)=0$ almost surely under each $P_x$ and $Y(0)=0$ almost surely under $P_Y$, so the raw test returns $0$ under all those laws.

For $x=i(v)$ in the stated null ball, [F7] gives $⟨Kx,\mathrm{\Phi }⟩=⟨x,K^{\ast }\mathrm{\Phi }{⟩}_X=⟨x,h{⟩}_X+r_{\mathrm{\Phi }}(v)\le \mathrm{\mid }{r}_{\mathrm{\Phi }}(v)\mathrm{\mid }\le \mathrm{\parallel }{r}_{\mathrm{\Phi }}{\mathrm{\parallel }}_{V^{\ast }}$. Thus the calibrated threshold is at least $⟨Kx,\mathrm{\Phi }⟩+ϵ\mathrm{\parallel }\mathrm{\Phi }\mathrm{\parallel }{z}_{1-\alpha }$. Applying step 1.1 to that threshold and using [F3] bounds the rejection probability by $1-G(z_{1-\alpha })=\alpha$. The same substitution gives the exact stated power formula at every $x$. If $\mathrm{\Phi }=0$, then $r_0(v)=-⟨i(v),h⟩$ and its norm is nonnegative, so the strict comparison is $0>\mathrm{\parallel }{r}_0{\mathrm{\parallel }}_{V^{\ast }}$, which is false; the level and power are zero.

For each ${\lambda }_j>0$, the scalar quadratic $\mathrm{\mid }\sqrt{{\lambda }_j}\xi -h_j{\mathrm{\mid }}^2+{ϵ}^2\mathrm{\mid }\xi {\mathrm{\mid }}^2\mathrm{/}{c}_j$ is minimized at ${\xi }_j=\sqrt{{\lambda }_j}{c}_j{h}_j\mathrm{/}({\lambda }_j{c}_j+{ϵ}^2)$. These coefficients define a vector in $H$ and lie in the inverse-square-root domain because $\mathrm{\mid }{\xi }_j{\mathrm{\mid }}^2\mathrm{/}{c}_j={\lambda }_j{c}_j\mathrm{\mid }{h}_j{\mathrm{\mid }}^2\mathrm{/}({\lambda }_j{c}_j+{ϵ}^2{)}^2\le \mathrm{\mid }{h}_j{\mathrm{\mid }}^2\mathrm{/}(4{ϵ}^2)$, using $t\mathrm{/}(t+{ϵ}^2{)}^2\le 1\mathrm{/}(4{ϵ}^2)$ for $t\ge 0$; Parseval makes the bound summable. The corresponding vector is $KSRSh={\mathrm{\Phi }}_h$, since $SRS$ has coefficient $c_j{h}_j\mathrm{/}({\lambda }_j{c}_j+{ϵ}^2)$ on $v_j$. Each coordinate minimizes its summand, while the ${\lambda }_j=0$ terms are fixed and the orthogonal observation component is irrelevant, so ${\mathrm{\Phi }}_h$ minimizes $J_h$. Finally [F7] and the diagonal action of $B$ show that $K^{\ast }{\mathrm{\Phi }}_h$ has coefficient ${\lambda }_j{c}_j{h}_j\mathrm{/}({\lambda }_j{c}_j+{ϵ}^2)$, which is the coefficient of $B(B+{ϵ}^{2}I{)}^{-1}h$.