Posterior contraction in Gaussian sequence inversion — Examples and Counterexamples

Given: AC and Countable Choice; $\alpha ,\beta ,p,L>0$; $0<ϵ\le 1$; the power ellipsoid and Gaussian prior in the Example; and the exact posterior moment formula for the coordinate model.

AC is the declared assumption for the Gaussian product posterior construction. The Axiom of Choice

Countable Choice is inherited from the compact-SVD sequence model. The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$)

The posterior coordinate mean and variance are ${\lambda }_j{Y}_j\mathrm{/}{s}_j$ and $v_j$, and the expected posterior squared error is the displayed nonnegative sum; posterior tail mass is bounded by that sum divided by the squared radius. Exact posterior second moment and contraction bound for a Gaussian sequence inverse problem

A rate is established when expected posterior mass outside every diverging multiple tends to zero under the fixed-truth law. Posterior contraction rate in a Gaussian sequence inverse experiment

The Gaussian sequence inverse model has ${\theta }^{†}\in {\mathrm{\ell }}^2$, positive singular values, product-Borel observations, and independent standard-normal noise coordinates. Gaussian sequence inverse model

For positive bases, real powers satisfy $a^x=\exp (x\log a)$ and the ordinary exponent laws. Real powers for positive bases, with the zero-base positive-exponent convention, The exponent, product, quotient, and iterated-power laws for positive real bases and real exponents

For every $q>0$, the geometric series ${\sum }_{k\ge 0}{2}^{-2qk}$ converges and equals $(1-2^{-2q}{)}^{-1}$. For $\mathrm{\mid }r\mathrm{\mid }<1$, ${\sum }_{k\ge 0}{r}^k=1\mathrm{/}(1-r)$, and for $\mathrm{\mid }r\mathrm{\mid }\ge 1$ the series diverges

The canonical real square-summable sequence space $H={\mathrm{\ell }}^2$ is complete, hence a real Hilbert space. Positive-index coordinates are identified with the ${\mathbb{N}}_0$-indexed sequence space by the isometry $(x_j{)}_{j\ge 1}\mapsto (x_{n+1}{)}_{n\ge 0}$. The real square-summable sequence space is complete The set of finite-support sequences with rational coordinates is countable and dense: truncate any square-summable sequence so the tail norm is arbitrarily small, then approximate the finitely many retained coordinates by rationals. Hence $H$ is separable. For the diagonal covariance, positivity and self-adjointness follow from $⟨\mathrm{\Gamma }x,x⟩={\sum }_j{\tau }_j^2{x}_j^2\ge 0$ and $⟨\mathrm{\Gamma }x,y⟩=⟨x,\mathrm{\Gamma }y⟩$. Finite-rank truncations converge in operator norm, so $\mathrm{\Gamma }$ is compact; its diagonal rank-one expansion has nuclear sum ${\sum }_j{\tau }_j^2<\mathrm{\infty }$, so it is trace class. A Hilbert space with a given orthonormal basis is ${\mathrm{\ell }}^2$ of the index set, Separability: the existence of an at most countable dense subset, Bounded finite rank operators are compact, Norm limit of compact operators is compact, Nuclear series characterizes trace norm

A positive self-adjoint trace-class covariance on a separable real Hilbert space defines a Borel centered Hilbert-valued Gaussian random element with that covariance. Positive trace-class covariance defines a Hilbert Gaussian

A finite vector is multivariate normal when all its linear projections are normal. Multivariate normal law, including singular covariance

Finite jointly Gaussian blocks with zero cross-covariance are independent, including for singular covariance matrices. Uncorrelated jointly Gaussian blocks are independent

The notation $N(m,{\sigma }^2)$ denotes the law of a normal variable with mean $m$ and variance ${\sigma }^2$. Standard normal and normal laws

For positive bases, $a>1$ makes $x\mapsto a^x$ increasing, $0<a<1$ makes it decreasing, and for $t>0$ the map $a\mapsto a^t$ is increasing while $a\mapsto a^{-t}$ is decreasing, since $a^{-t}=1\mathrm{/}{a}^t$. In particular, for $q>0$, $0<2^{-2q}<1$, because $\log 2>\log 1=0$ and $\exp$ is positive and increasing. For $0<ϵ<1$, $\log ϵ<0$, so $u\ge v$ implies ${ϵ}^u\le {ϵ}^v$. Finally, for $\eta >0$ and any $M>0$, if $0<ϵ<\exp (-M\mathrm{/}\eta )$ then $\eta \log ϵ<-M$, so ${ϵ}^{\eta }<\exp (-M)$; the latter tends to zero as $M\to \mathrm{\infty }$. These facts use $a^x=\exp (x\log a)$, strict increase of $\log$ and $\exp$, $\log 1=0$, positivity of $\exp$, and its limit at $-\mathrm{\infty }$. Real powers for positive bases, with the zero-base positive-exponent convention, The natural logarithm as the inverse of the exponential function, The real exponential function and the number $e$ by a power series, Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm, The exponential function is strictly increasing, The exponential tends to $+\mathrm{\infty }$ at $+\mathrm{\infty }$ and to $0$ at $-\mathrm{\infty }$, The exponential is positive and satisfies $\exp (-x)=1\mathrm{/}\exp (x)$

Given: The hypotheses of the Example and facts [A1], [F1]–[F11].

Given: AC and Countable Choice; $\alpha ,\beta ,p,L>0$; $0<ϵ<1\mathrm{/}{e}^{\alpha +p}$; the exponential ellipsoid and independent Gaussian prior in the Example; and the exact posterior moment formula for the coordinate model.

AC is the declared assumption for the Gaussian product posterior construction. The Axiom of Choice

Countable Choice is inherited from the compact-SVD sequence model. The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$)

The posterior coordinate mean and variance are ${\lambda }_j{Y}_j\mathrm{/}{s}_j$ and $v_j$, and the expected posterior squared error is the displayed nonnegative sum; posterior tail mass is bounded by that sum divided by the squared radius. Exact posterior second moment and contraction bound for a Gaussian sequence inverse problem

A rate is established when expected posterior mass outside every diverging multiple tends to zero under the fixed-truth law. Posterior contraction rate in a Gaussian sequence inverse experiment

The Gaussian sequence inverse model has ${\theta }^{†}\in {\mathrm{\ell }}^2$, positive singular values, product-Borel observations, and independent standard-normal noise coordinates. Gaussian sequence inverse model

For positive bases, real powers obey $a^x=\exp (x\log a)$ and the exponent laws; the natural logarithm is the inverse of the exponential and $\log 1=0$, so $\log e=1$ and $\exp (0)=1$. Real powers for positive bases, with the zero-base positive-exponent convention, The exponent, product, quotient, and iterated-power laws for positive real bases and real exponents, The natural logarithm as the inverse of the exponential function, Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm

The exponential is positive, strictly increasing, tends to infinity at infinity, and obeys $\exp (x+y)=\exp (x)\exp (y)$. The real exponential function and the number $e$ by a power series, The exponential function is strictly increasing, The exponential tends to $+\mathrm{\infty }$ at $+\mathrm{\infty }$ and to $0$ at $-\mathrm{\infty }$, The exponential addition formula $\exp (x+y)=\exp (x)\exp (y)$, The exponential is positive and satisfies $\exp (-x)=1\mathrm{/}\exp (x)$

If $0<r<1$, the geometric series ${\sum }_{k\ge 0}{r}^k$ converges to $(1-r{)}^{-1}$. For $\mathrm{\mid }r\mathrm{\mid }<1$, ${\sum }_{k\ge 0}{r}^k=1\mathrm{/}(1-r)$, and for $\mathrm{\mid }r\mathrm{\mid }\ge 1$ the series diverges

The canonical real square-summable sequence space $H={\mathrm{\ell }}^2$ is complete, hence a real Hilbert space. Positive-index coordinates are identified with the ${\mathbb{N}}_0$-indexed sequence space by the isometry $(x_j{)}_{j\ge 1}\mapsto (x_{n+1}{)}_{n\ge 0}$. The real square-summable sequence space is complete The set of finite-support sequences with rational coordinates is countable and dense: truncate any square-summable sequence so the tail norm is arbitrarily small, then approximate the finitely many retained coordinates by rationals. Hence $H$ is separable. For the diagonal covariance, positivity and self-adjointness follow from $⟨\mathrm{\Gamma }x,x⟩={\sum }_j{\tau }_j^2{x}_j^2\ge 0$ and $⟨\mathrm{\Gamma }x,y⟩=⟨x,\mathrm{\Gamma }y⟩$. Finite-rank truncations converge in operator norm, so $\mathrm{\Gamma }$ is compact; its diagonal rank-one expansion has nuclear sum ${\sum }_j{\tau }_j^2<\mathrm{\infty }$, so it is trace class. A Hilbert space with a given orthonormal basis is ${\mathrm{\ell }}^2$ of the index set, Separability: the existence of an at most countable dense subset, Bounded finite rank operators are compact, Norm limit of compact operators is compact, Nuclear series characterizes trace norm

A positive self-adjoint trace-class covariance on a separable real Hilbert space defines a Borel centered Hilbert-valued Gaussian random element with that covariance. Positive trace-class covariance defines a Hilbert Gaussian

A finite vector is multivariate normal when all its linear projections are normal. Multivariate normal law, including singular covariance

Finite jointly Gaussian blocks with zero cross-covariance are independent, including for singular covariance matrices. Uncorrelated jointly Gaussian blocks are independent

The notation $N(m,{\sigma }^2)$ denotes the law of a normal variable with mean $m$ and variance ${\sigma }^2$. Standard normal and normal laws

For positive bases, $a>1$ makes $x\mapsto a^x$ increasing, $0<a<1$ makes it decreasing, and for $t>0$ the map $a\mapsto a^t$ is increasing while $a\mapsto a^{-t}$ is decreasing, since $a^{-t}=1\mathrm{/}{a}^t$. For $0<ϵ<1$, $\log ϵ<0$, so $u\ge v$ implies ${ϵ}^u\le {ϵ}^v$. Finally, for $\eta >0$ and any $M>0$, if $0<ϵ<\exp (-M\mathrm{/}\eta )$ then $\eta \log ϵ<-M$, so ${ϵ}^{\eta }<\exp (-M)$; the latter tends to zero as $M\to \mathrm{\infty }$. These follow from [F4] and [F5]: $a^x=\exp (x\log a)$, $\log$ and $\exp$ are strictly increasing, $\log 1=0$, and $\exp (x)\to 0$ as $x\to -\mathrm{\infty }$. Real powers for positive bases, with the zero-base positive-exponent convention, The natural logarithm as the inverse of the exponential function, Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm, The exponential function is strictly increasing, The exponential tends to $+\mathrm{\infty }$ at $+\mathrm{\infty }$ and to $0$ at $-\mathrm{\infty }$, The exponential is positive and satisfies $\exp (-x)=1\mathrm{/}\exp (x)$

Given: The hypotheses of the Example and facts [A1], [F1]–[F12].

Given: AC and Countable Choice; the positive-noise sequence inverse experiment; its standard basis vector $e_1=(1,0,0,\dots )$; and the posterior contraction definition.

AC and Countable Choice are assumed by the Gaussian sequence model and its product-law theorem. The Axiom of Choice, The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$)

The data coordinates are $Y_j=s_j{\theta }_j+ϵ{\xi }_j$ with positive $s_j$ and independent standard-normal noise. Gaussian sequence inverse model

The conjugate contraction theorem assumes ${\tau }_j>0$ and ${\sum }_j{\tau }_j^2<\mathrm{\infty }$; its conclusion is not stated for every Gaussian prior without those hypotheses. Exact posterior second moment and contraction bound for a Gaussian sequence inverse problem

The law $N(0,0)$ is the point mass ${\delta }_0$. Standard normal and normal laws

A nondegenerate finite-dimensional Gaussian with covariance ${ϵ}^{2}I$ has density proportional to $\exp (-\mathrm{\parallel }y-m{\mathrm{\parallel }}_2^2\mathrm{/}(2{ϵ}^2))$. A positive-definite multivariate normal law has its Gaussian density

Finite measures agreeing on a generating cylinder $\pi$-system and on the whole space agree on the generated product Borel sigma-algebra. Finite measures agreeing on a generating pi-system and on the whole space are equal

The real exponential is strictly positive. The exponential is positive and satisfies $\exp (-x)=1\mathrm{/}\exp (x)$

The posterior contraction condition requires the expected posterior mass outside every diverging multiple of a rate to tend to zero under the fixed-truth data law. Posterior contraction rate in a Gaussian sequence inverse experiment