Banach Gaussian priors, RKHS support, and concentration — Examples

For any finite list of indices $t_i$ and real coefficients $a_i$, ${\sum }_{i,j}{a}_i{a}_{j}k(t_i,t_j)={\sum }_t({\sum }_{i:t_i=t}{a}_i{)}^2\ge 0$, so $k$ is positive semidefinite. By [F1], its zero-mean cylinder law has marginal $N_d(0,I_d)$ on every set of $d$ distinct coordinates. The independent standard-normal vector in [F2] also has law $N_d(0,I_d)$. Their characteristic functions agree by [F6], whose uniqueness assertion identifies the two laws. Thus every finite family of distinct coordinates is independent and standard normal.

Fix $N$. For $M\ge N$, finite-coordinate independence gives $P(\mathrm{\mid }{Z}_k\mathrm{\mid }\le 1\text{ for all }N\le k\le M)=(1-q{)}^{M-N+1}$. These events decrease as $M\to \mathrm{\infty }$, so [F3] shows $P(\mathrm{\mid }{Z}_k\mathrm{\mid }\le 1\text{ for every }k\ge N)=0$. The event that only finitely many $\mathrm{\mid }{Z}_k\mathrm{\mid }$ exceed $1$ is the countable union over $N$ of these null events; [F4] makes that union null, so almost surely infinitely many coordinates satisfy $\mathrm{\mid }{Z}_k\mathrm{\mid }>1$.

Infinitely many terms of ${\sum }_k{Z}_k^2$ exceed $1$, hence the sum diverges almost surely. The set ${\mathrm{\ell }}^2\subset {\mathbb{R}}^{\mathbb{N}}$ is cylinder-measurable because it equals ${\bigcup }_{m\ge 1}{\bigcap }_{N\ge 1}\{{\sum }_{k\le N}{x}_k^2\le m\}$; its cylinder probability is therefore zero by step 2.1. If an ${\mathrm{\ell }}^2$-valued Borel random element had these finite-coordinate laws, its coordinate map into ${\mathbb{R}}^{\mathbb{N}}$ would be measurable for the cylinder sigma-algebra, so its pushforward would agree with the given cylinder law on every finite-coordinate cylinder. By [F5], the two probability measures are equal; but the pushforward assigns probability one to ${\mathrm{\ell }}^2$, contradicting its zero cylinder probability. Thus no Borel ${\mathrm{\ell }}^2$-valued random element, and in particular no tight Borel ${\mathrm{\ell }}^2$ prior, has these finite-coordinate laws.

With $v=(1,2)$, the mean is zero and $E[X_i{X}_j]=v_i{v}_j$. The kernel $k(i,j)=v_i{v}_j$ is positive semidefinite because for all $a_1,a_2$, ${\sum }_{i,j}{a}_i{a}_{j}k(i,j)=(a_1+2a_2{)}^2\ge 0$. Thus [F1] gives the same possibly singular Gaussian process law as the explicit vector.

The map $z\mapsto (z,2z)$ is continuous, so its pushforward of the standard normal law is a Borel probability on $B$. For every positive integer $R$, the events $\{\mathrm{\mid }Z\mathrm{\mid }\le R\}$ increase to the whole space; continuity from below [F4] therefore lets us choose $R$ with $P(\mathrm{\mid }Z\mathrm{\mid }\le R)>1-\delta$. The image segment $\{(z,2z):\mathrm{\mid }z\mathrm{\mid }\le R\}$ is compact and has probability above $1-\delta$, so the law is tight.

For ${\mathrm{\ell }}_a(x)=a_1{x}_1+a_2{x}_2$, ${\mathrm{\ell }}_a(X)=(a_1+2a_2)Z$, and [F2] gives $C{\mathrm{\ell }}_a=E[{\mathrm{\ell }}_a(X)X]=(a_1+2a_2)(1,2)\mathrm{.}$ The range is exactly $\mathrm{span}\{(1,2)\}$. If $h=c(1,2)$, then every representative with $a_1+2a_2=c$ has covariance norm squared $E[{\mathrm{\ell }}_a(X{)}^2]=c^2$, so $\mathrm{\parallel }h{\mathrm{\parallel }}_H=\mathrm{\mid }c\mathrm{\mid }$.

The law is concentrated on this line. Every open ball around $c(1,2)$ meets the line in an interval of positive normal probability, while a point outside the line has a sufficiently small ball disjoint from it. Hence the support is the line, equal to the RKHS. Since $\mathrm{\parallel }X{\mathrm{\parallel }}^2=5Z^2$, [F2] and [F5] give $E{e}^{a\mathrm{\parallel }X{\mathrm{\parallel }}^2}=E{e}^{5a{Z}^2}=\frac{1}{\sqrt{2\pi }}{\int }_{\mathbb{R}}{e}^{-(1\mathrm{/}2-5a)z^2}dz=(1-10a{)}^{-1\mathrm{/}2}<\mathrm{\infty }$ for $0<a<1\mathrm{/}10$.