Exact posterior second moment and contraction bound for a Gaussian sequence inverse problem

For each fixed coordinate $j$, apply [F2] with one-dimensional prior variance $C_0={\tau }_j^2$, observation map $A=s_j$, and noise variance $\mathrm{\Gamma }={ϵ}^2$. Then $A{C}_0{A}^T+\mathrm{\Gamma }=s_j^2{\tau }_j^2+{ϵ}^2>0$, and [F2] gives posterior mean ${\tau }_j^2{s}_j{Y}_j\mathrm{/}({ϵ}^2+{\tau }_j^2{s}_j^2)={\lambda }_j{Y}_j\mathrm{/}{s}_j$ and variance ${\tau }_j^2-{\tau }_j^4{s}_j^2\mathrm{/}({ϵ}^2+{\tau }_j^2{s}_j^2)=v_j$.

For every finite coordinate set $J$, the prior/noise pairs indexed by $J$ are independent of the pairs outside $J$: independence of the prior element from the noise element and independence within each coordinate family factor the probabilities of finite coordinate rectangles into the product of the pair marginals, and [F5] extends this factorization from the generating rectangles to the corresponding product-Borel events. Each $Y_j$ is a measurable function of its pair $({\theta }_j,{\xi }_j)$, so $({\theta }_J,Y_J)$ is independent of $Y_{J^c}$. For the finite block, put $D_{\tau }=\mathrm{diag}({\tau }_j^2{)}_{j\in J}$ and $D_s=\mathrm{diag}(s_j{)}_{j\in J}$. [F2] gives posterior covariance $D_{\tau }-D_{\tau }{D}_s^T(D_s{D}_{\tau }{D}_s^T+{ϵ}^{2}I{)}^{-1}{D}_s{D}_{\tau }=\mathrm{diag}(v_j{)}_{j\in J},$ and posterior mean with coordinates ${\lambda }_j{Y}_j\mathrm{/}{s}_j$. Since each $v_j>0$, the Gaussian coordinates have zero cross-covariance, and repeated application of [F10] shows they are independent. Thus the finite-block conditional law is the product of the scalar laws from step 1.1. Independence of the complement makes this also the conditional law given the full data: the conditional identity first holds on data rectangles by finite-block conditioning and independence, and [F5] extends it to all product-Borel data events. The product Gaussian law $Q_y={⨂}_{j\ge 1}N({\lambda }_j{y}_j\mathrm{/}{s}_j,v_j)$ exists on ${\mathbb{R}}^{{\mathbb{N}}_{>0}}$ by [F3]. On finite-coordinate cylinders its evaluations are Borel functions of $y$; the class of Borel sets $B$ for which $y\mapsto Q_y(B)$ is measurable is a Dynkin system containing the generating cylinder $\pi$-system, so [F6] makes $Q$ a probability kernel. The same finite-block conditional identity holds on data cylinders, and [F5] applied successively in the data and parameter coordinates extends it to all product-Borel events. Thus $Q_y$ is the prior-predictive conditional law on the product coordinate space.

Put $m_j(y)={\lambda }_j{y}_j\mathrm{/}{s}_j$ and $Y_0=\{y:{\sum }_j{m}_j(y{)}^2<\mathrm{\infty }\}$. This is Borel because each $m_j$ is coordinate-measurable and ${\mathrm{\ell }}^2={\bigcup }_{k\ge 1}{\bigcap }_{N\ge 1}\{x:{\sum }_{j=1}^N{x}_j^2\le k\}$. The norm-Borel sigma-algebra of ${\mathrm{\ell }}^2$ equals the trace of the product Borel sigma-algebra: finite-support rational sequences are countable and dense (truncate a square-summable sequence and approximate the retained coordinates), and for $a\in {\mathrm{\ell }}^2$ and $r>0$, $B_{{\mathrm{\ell }}^2}(a,r)={\mathrm{\ell }}^2\cap {\bigcup }_{\begin{array}{c}q\in \mathbb{Q}\\ 0<q<r^2\end{array}}{\bigcap }_{N\ge 1}\{x\in {\mathbb{R}}^{{\mathbb{N}}_{>0}}:{\sum }_{j=1}^N(x_j-a_j{)}^2\le q\}$. Thus every norm ball, and hence every norm-open set, is trace product-Borel; conversely the coordinate maps on normed ${\mathrm{\ell }}^2$ are continuous. Therefore the restriction of $Q_y$ to ${\mathrm{\ell }}^2$ is measurable on its norm-Borel sets. Since $v_j\le {\tau }_j^2$, [F7] gives $\int {\sum }_j{x}_j^2{Q}_y(dx)={\sum }_j(m_j(y{)}^2+v_j)<\mathrm{\infty }$ on $Y_0$, so $Q_y({\mathrm{\ell }}^2)=1$ there. Under the prior-predictive law, $\mathbb{E}{m}_j(Y{)}^2={\lambda }_j^2{\tau }_j^2+{\lambda }_j^2{ϵ}^2\mathrm{/}{s}_j^2={\lambda }_j{\tau }_j^2\le {\tau }_j^2$; under the fixed truth, $m_j(Y)={\lambda }_j{\theta }_j^{†}+{\lambda }_jϵ{\xi }_j\mathrm{/}{s}_j$ and ${\mathbb{E}}_{{\theta }^{†}}{m}_j(Y{)}^2={\lambda }_j^2({\theta }_j^{†}{)}^2+{\lambda }_j^2{ϵ}^2\mathrm{/}{s}_j^2\le ({\theta }_j^{†}{)}^2+{\tau }_j^2$. Applying [F7] to the partial sums shows $Y_0$ has full measure under both laws. Restrict $Q_y$ to ${\mathrm{\ell }}^2$ on $Y_0$ and set it equal to ${\delta }_0$ outside $Y_0$; this is a posterior version on ${\mathrm{\ell }}^2$ and is defined for fixed-truth-almost every observation.

For $y\in Y_0$, the product posterior has coordinate mean $m_j(y)$ and variance $v_j$, so its expected squared coordinate error is $(m_j(y)-{\theta }_j^{†}{)}^2+v_j$. The partial sums of these nonnegative errors increase to $\mathrm{\parallel }\theta -{\theta }^{†}{\mathrm{\parallel }}_2^2$; [F7] therefore gives $\int \mathrm{\parallel }\theta -{\theta }^{†}{\mathrm{\parallel }}_2^2{\mathrm{\Pi }}_{ϵ}(d\theta \mid y)={\sum }_j((m_j(y)-{\theta }_j^{†}{)}^2+v_j)$, with the value $+\mathrm{\infty }$ allowed.

Under the fixed truth, $m_j(Y)-{\theta }_j^{†}=-(1-{\lambda }_j){\theta }_j^{†}+{\lambda }_jϵ{\xi }_j\mathrm{/}{s}_j$. The noise has mean zero and variance one, so its cross term has expectation zero and ${\mathbb{E}}_{{\theta }^{†}}(m_j(Y)-{\theta }_j^{†}{)}^2=(1-{\lambda }_j{)}^2({\theta }_j^{†}{)}^2+{\lambda }_j^2{ϵ}^2\mathrm{/}{s}_j^2$. The identity $v_j{s}_j^2\mathrm{/}{ϵ}^2={\lambda }_j$ gives ${\lambda }_j^2{ϵ}^2\mathrm{/}{s}_j^2=v_j^2{s}_j^2\mathrm{/}{ϵ}^2$. Taking expectations in step 4.1 and applying [F7] to the nonnegative partial sums proves the displayed exact series identity.

For each fixed data sequence, apply [F8] to the posterior probability measure with $f(\theta )=\mathrm{\parallel }\theta -{\theta }^{†}{\mathrm{\parallel }}_2^2$ and threshold $R^2>0$. This gives posterior tail mass at most $R^{-2}$ times the posterior second moment. Taking fixed-truth expectation and using step 5.1 yields the stated bound. If $S_{ϵ}({\theta }^{†})\le C{r}_{ϵ}^2$, substitute $R=M_{ϵ}{r}_{ϵ}$ to obtain expected tail mass at most $C\mathrm{/}{M}_{ϵ}^2\to 0$ for every $M_{ϵ}\to \mathrm{\infty }$; [F9] identifies $r_{ϵ}$ as a posterior contraction rate.