Posterior and marginal-likelihood formulas for the Gaussian sequence prior

By [F1]–[F2], the independent prior coordinates and independent normal errors can be realized on countable product probability spaces. Fix one coordinate and put $s_i=i^{-r}$. By [F4], [F9] and [F11], the prior and sampling densities with respect to Lebesgue measure are $g_i(t)=(2\pi s_i{)}^{-1\mathrm{/}2}{e}^{-t^2\mathrm{/}(2s_i)},q_i(x\mid t)=\sqrt{\frac{n}{2\pi }}{e}^{-n(x-{\kappa }_{i}t{)}^2\mathrm{/}2}\mathrm{.}$ Their product has exponent $-\frac{1}{2}[(s_i^{-1}+n{\kappa }_i^2)t^2-2n{\kappa }_{i}xt+n{x}^2]$. Completing the square in $t$ gives conditional variance $v_i=(s_i^{-1}+n{\kappa }_i^2{)}^{-1}$ and conditional mean $m_i(x)=v_{i}n{\kappa }_{i}x$. Integrating the same product over $t$ gives the prior-predictive density $\frac{\exp \{-x^2\mathrm{/}[2(n^{-1}+{\kappa }_i^2{s}_i)]\}}{\sqrt{2\pi (n^{-1}+{\kappa }_i^2{s}_i)}}\mathrm{.}$ This is the density of $N(0,n^{-1}+{\kappa }_i^2{s}_i)$ by [F4], [F9] and [F11]. In the procedure's parameter range, [F8] identifies the same coordinate update.

The scalar posterior densities in step 1.1 are probability densities by [F4], [F9] and [F11]. For each fixed data sequence, [F2] defines the product posterior measure $K(x,\cdot )$. On a finite-coordinate parameter cylinder $A$, $K(x,A)$ is a finite product of normal probabilities, hence is Borel in $x$. The scalar Bayes identity and independence of the coordinate pairs give the conditional-law identity for such $A$ and every finite-coordinate data cylinder. For fixed $A$, both sides, as functions of the data event, are finite measures; the scalar Bayes identity with the full data event shows that both have total mass equal to the prior probability of $A$. If that mass is positive, normalize both measures and apply [F6]; if it is zero, both measures are identically zero. The data cylinders generate the data sigma-algebra by [F3], so the identity holds for every data event. The class of parameter sets $A$ for which $K(\cdot ,A)$ is Borel and the identity holds for every data event contains the parameter-cylinder $\pi$-system. It is a Dynkin system because complements use $1-K(x,A)$ and differences of finite measures, while countable disjoint unions use pointwise limits of measurable partial sums and countable additivity of the joint measures. A second use of [F3] and [F6] proves the identity for every product-measurable $A$. Therefore $K$ is a posterior probability kernel.

Write $a_i=n{\kappa }_i^2{s}_i$. The density of the marginal $N(0,n^{-1}(1+a_i))$ coordinate relative to $N(0,n^{-1})$ is $L_i(x_i)=(1+a_i{)}^{-1\mathrm{/}2}\exp \{n{a}_i{x}_i^2\mathrm{/}[2(1+a_i)]\}$. Since ${\sum }_i{a}_i\le n{C}^2{\sum }_i{i}^{-1-2\alpha -2p}<\mathrm{\infty }$, [F5] and [F10] give ${\sum }_i{a}_i{Z}_i^2<\mathrm{\infty }$ almost surely under $Q_{n,0}$, where $Z_i=\sqrt{n}{X}_i$. Hence ${\sum }_i\mathrm{\mid }\log L_i(X_i)\mathrm{\mid }<\mathrm{\infty }$ there. Choose $J$ so $a_i<1$ for $i>J$. Direct Gaussian integration gives $E_{Q_{n,0}}{L}_i^2=(1-a_i^2{)}^{-1\mathrm{/}2}$ for $i>J$, and ${\prod }_{i>J}(1-a_i^2{)}^{-1\mathrm{/}2}<\mathrm{\infty }$ because ${\sum }_i{a}_i^2<\mathrm{\infty }$. The finite tail products are mean-one martingales and are Cauchy in $L^2$: for $m>k>J$, independence gives $E\mathrm{\mid }{\prod }_{i=J+1}^m{L}_i-{\prod }_{i=J+1}^k{L}_i{\mathrm{\mid }}^2=E({\prod }_{i=J+1}^k{L}_i^2)\{{\prod }_{i=k+1}^m(1-a_i^2{)}^{-1\mathrm{/}2}-1\}$, which tends uniformly to zero as $k\to \mathrm{\infty }$. The finite initial product is independent of the tail and has mean one, so multiplication preserves $L^1$ convergence. By absolute convergence of the log series, the nonnegative limit is ${\prod }_i{L}_i=\exp ({\sum }_i\log L_i)$ and has expectation one. It agrees with the marginal product law on every finite cylinder, so [F6] identifies it as $d{Q}_{n,\alpha }\mathrm{/}d{Q}_{n,0}$. Taking logarithms and substituting $s_i=i^{-r}$ gives the displayed ${\mathrm{\ell }}_n(\alpha ;x)$.

Under $P_{{\theta }_0}^{(n)}$, [F10] and the independence in the model give $E{X}_i^2={\kappa }_i^2{\theta }_{0,i}^2+n^{-1}$. The sum ${\sum }_i{a}_{i}E(n{X}_i^2)$ is finite: its noise part is ${\sum }_i{a}_i$, and its signal part is $n^2{\sum }_i{\kappa }_i^4{i}^{-r}{\theta }_{0,i}^2\le n^2{C}^4{\sum }_i{\theta }_{0,i}^2<\mathrm{\infty }$. Thus the same absolute-convergence bound for ${\sum }_i\mathrm{\mid }\log L_i(X_i)\mathrm{\mid }$ holds under the true sampling law.

By [F5] and [F10], the prior has expected squared norm ${\sum }_i{i}^{-r}<\mathrm{\infty }$, hence belongs to ${\mathrm{\ell }}^2$ almost surely. The posterior variances satisfy ${\sum }_i{v}_i\le {\sum }_i{i}^{-r}<\mathrm{\infty }$. For its mean, write $m_i(X_i)=w_i{\theta }_{0,i}+\sqrt{n}{\kappa }_i(i^r+n{\kappa }_i^2{)}^{-1}{Z}_i$, where $w_i=n{\kappa }_i^2\mathrm{/}(i^r+n{\kappa }_i^2)\le 1$. The cross term has mean zero because $E{Z}_i=0$ by [F10], so $E_{{\theta }_0}^{(n)}{\sum }_i{m}_i(X_i{)}^2\le {\sum }_i{\theta }_{0,i}^2+n{C}^2{\sum }_i{i}^{-2r-2p}<\mathrm{\infty }\mathrm{.}$ Therefore the posterior mean is in ${\mathrm{\ell }}^2$ almost surely; for almost every such observation its posterior expected squared norm is ${\sum }_i(m_i(X_i{)}^2+v_i)<\mathrm{\infty }$, so the posterior gives ${\mathrm{\ell }}^2$ mass one.