Small-ball MAP and Cameron–Martin geometry

Choose the norming family from [F3]. Fix $r>0$, $h\in E$, and $n\ge 1$. Write $Y_n=({\mathrm{\ell }}_1(W),\dots ,{\mathrm{\ell }}_n(W))$, let ${\mathrm{\Sigma }}_n$ be its covariance matrix, and put $a_n=({\mathrm{\ell }}_1(ih),\dots ,{\mathrm{\ell }}_n(ih))$. The vector $Y_n$ is centered Gaussian by [F1]. If $c\in \ker {\mathrm{\Sigma }}_n$, then ${\sum }_{j\le n}{c}_j{\mathrm{\ell }}_j(W)$ has variance zero and hence is zero almost surely. For ${\mathrm{\ell }}_c={\sum }_{j\le n}{c}_j{\mathrm{\ell }}_j$, this gives $C{\mathrm{\ell }}_c=0$ and $[{\mathrm{\ell }}_c]=0$ in $E$; therefore [F2] gives $c\cdot a_n={\mathrm{\ell }}_c(ih)=⟨h,[{\mathrm{\ell }}_c]{⟩}_E=0$. Thus $a_n\perp \ker {\mathrm{\Sigma }}_n$, so $a_n\in S_n:=\mathrm{ran}({\mathrm{\Sigma }}_n)$, the support subspace of $Y_n$.

In the finite-dimensional space $S_n$, the restriction of $Y_n$ is a nondegenerate Gaussian vector. Let $Q_n=[-r,r{]}^n$ and $D_n=Q_n\cap S_n$, a closed symmetric convex set. Since $a_n\in S_n$, the event $Y_n-a_n\in Q_n$ is the translate by $a_n$ of $D_n$ inside $S_n$. An invertible linear change of coordinates turns the centered law on $S_n$ into a nondegenerate Gaussian law and preserves symmetry and convexity of $D_n$. Apply [F4] to the bounded loss $1_{D_n^c}$ in those coordinates: its sublevel sets are $D_n$ and the whole space, so it is symmetric and quasi-convex. This yields $\mathbb{P}(Y_n-a_n\in Q_n)\le \mathbb{P}(Y_n\in Q_n)\mathrm{.}$ If $S_n=\{0\}$, then $Y_n=a_n=0$ almost surely and the same inequality is equality.

Define $A_n(h)=\{{\max }_{j\le n}\mathrm{\mid }{\mathrm{\ell }}_j(W-ih)\mathrm{\mid }\le r\}$. Step 2.1 says $\mathbb{P}(A_n(h))\le \mathbb{P}(A_n(0))$. The sets $A_n(h)$ decrease with $n$, and [F3] gives ${\bigcap }_n{A}_n(h)=\{\mathrm{\parallel }W-ih{\mathrm{\parallel }}_B\le r\}$; likewise ${\bigcap }_n{A}_n(0)=\{\mathrm{\parallel }W{\mathrm{\parallel }}_B\le r\}$. Apply [F5] to both decreasing sequences and pass to their limits in the finite-dimensional inequalities. This proves the stated ball-mass bound.

Put $A_r={\bar{B}}_B(0,r)$. By [F4], $\mu (A_r)>0$ because it contains the open ball of radius $r$ about the support point $0$. Let $({\mathrm{\ell }}_j)$ be the norming family from [F6], and define $A_{r,n}={\bigcap }_{j\le n}\{\mathrm{\mid }{\mathrm{\ell }}_j(W)\mathrm{\mid }\le r\}$. The sets decrease to $A_r$. For fixed $n$, the vector $(z_s,{\mathrm{\ell }}_1(W),\dots ,{\mathrm{\ell }}_n(W))$ is centered Gaussian: [F3] makes $z_s$ an $L^2$ limit of continuous-linear Gaussian coordinates, so adjoining the finitely many displayed coordinates gives a centered Gaussian limit. Its law is invariant under simultaneous sign reversal, and $A_{r,n}$ is invariant under that reversal. Hence $\mathbb{E}[e^{z_s}{\mathbf{1}}_{A_{r,n}}]=\mathbb{E}[e^{-z_s}{\mathbf{1}}_{A_{r,n}}],\mathbb{E}[z_s{\mathbf{1}}_{A_{r,n}}]=0.$ The variables $e^{\pm z_s}$ are integrable by the Gaussian moment formula in [F3]. Dominated convergence as $n\to \mathrm{\infty }$ therefore gives conditional symmetry and conditional mean zero on $A_r$. Thus $1\le {\mathbb{E}}_{\mu }[e^{-z_s}\mid A_r]\le e^{\mathrm{\parallel }s{\mathrm{\parallel }}_E^2\mathrm{/}2}\mathrm{.}$ For the lower bound, conditional Jensen applies. For the upper bound, [F3] applied to $-s$ gives ${\mathbb{E}}_{\mu }[e^{-z_s}{\mathbf{1}}_{A_r}]=e^{\mathrm{\parallel }s{\mathrm{\parallel }}_E^2\mathrm{/}2}\mu ({\bar{B}}_B(is,r)),$ and [F5] bounds the last probability by $\mu (A_r)$.

For any $h\in E$, choose ${\mathrm{\ell }}_n\in B^{\ast }$ with $h_n:=C{\mathrm{\ell }}_n\to h$ in $E$, possible by the completion in [F2], and put $s_n=h-h_n$. The Cameron–Martin formula [F3], applied to the translate by $-h$, gives $\frac{\mu ({\bar{B}}_B(ih,r))}{\mu (A_r)}=e^{-\mathrm{\parallel }h{\mathrm{\parallel }}_E^2\mathrm{/}2}{\mathbb{E}}_{\mu }[e^{-z_h}\mid A_r]\mathrm{.}$ Since $z_{h_n}={\mathrm{\ell }}_n(W)$ almost surely and $z_h=z_{h_n}+z_{s_n}$, the bound $\mathrm{\mid }{\mathrm{\ell }}_n(W)\mathrm{\mid }\le r\mathrm{\parallel }{\mathrm{\ell }}_n{\mathrm{\parallel }}_{B^{\ast }}$ on $A_r$ and step 1.1 imply $e^{-r\mathrm{\parallel }{\mathrm{\ell }}_n{\mathrm{\parallel }}_{B^{\ast }}}\le {\mathbb{E}}_{\mu }[e^{-z_h}\mid A_r]\le e^{r\mathrm{\parallel }{\mathrm{\ell }}_n{\mathrm{\parallel }}_{B^{\ast }}}{e}^{\mathrm{\parallel }{s}_n{\mathrm{\parallel }}_E^2\mathrm{/}2}\mathrm{.}$ For fixed $n$, let $r\downarrow 0$; then let $n\to \mathrm{\infty }$, so $\mathrm{\parallel }{s}_n{\mathrm{\parallel }}_E\to 0$. The two bounds show ${\mathbb{E}}_{\mu }[e^{-z_h}\mid A_r]\to 1$.

Applying step 2.1 separately to $h_1$ and $h_2$ yields $\frac{\mu ({\bar{B}}_B(i{h}_1,r))}{\mu ({\bar{B}}_B(i{h}_2,r))}=e^{(\mathrm{\parallel }{h}_2{\mathrm{\parallel }}_E^2-\mathrm{\parallel }{h}_1{\mathrm{\parallel }}_E^2)\mathrm{/}2}\frac{\mathbb{E}[e^{-z_{h_1}}\mid A_r]}{\mathbb{E}[e^{-z_{h_2}}\mid A_r]}\mathrm{.}$ Both conditional expectations tend to $1$, so the ratio has the stated limit. Positivity of the denominator follows from [F4], since each closed ball contains the corresponding open neighborhood of a support point.

Choose $0<\epsilon <a$ with $a$ from [F3]. The lower bound gives $\mathrm{\Phi }(u)\ge M_{\epsilon }-\epsilon \mathrm{\parallel }u{\mathrm{\parallel }}_B^2$. Define the nonnegative Borel function $M_1(t)=\max \{0,\epsilon t^2-M_{\epsilon }\}$. Then $\mathrm{\Phi }(u)\ge -M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_B)$ and $e^{M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_B)}\le 1+e^{-M_{\epsilon }}{e}^{\epsilon \mathrm{\parallel }u{\mathrm{\parallel }}_B^2}\mathrm{.}$ The right side is $\mu$-integrable by [F3]. Since $\mathrm{\Phi }$ is locally Lipschitz it is finite and continuous on $B$. Apply [F2] with the single datum and $D=B$; the prior definition in [F1] makes $\mu$ a prior on $B$. Thus $0<Z<\mathrm{\infty }$ and $\mathrm{\Pi }$ is a probability.

Let $C_i=\mathrm{\parallel }i:E\to B\mathrm{\parallel }$. If $E=\{0\}$, $I$ has the sole candidate $0$ and is minimized there. Otherwise $C_i>0$. Choose $\epsilon <1\mathrm{/}(4C_i^2)$ and use the potential lower bound to obtain, for every $h\in E$, $I(ih)\ge M_{\epsilon }+\frac{1}{2}\mathrm{\parallel }h{\mathrm{\parallel }}_E^2-\epsilon \mathrm{\parallel }ih{\mathrm{\parallel }}_B^2\ge M_{\epsilon }+\frac{1}{4}\mathrm{\parallel }h{\mathrm{\parallel }}_E^2\mathrm{.}$ Since $I(0)=\mathrm{\Phi }(0)$ is finite, ${\inf }_{E}I$ is finite and any minimizing sequence is bounded in $E$.

Fix $h\in E$. Local Lipschitz continuity at $ih$ supplies constants $L_h<\mathrm{\infty }$ and ${\rho }_h>0$ such that $\mathrm{\mid }\mathrm{\Phi }(u)-\mathrm{\Phi }(ih)\mathrm{\mid }\le L_{h}r$ whenever $\mathrm{\parallel }u-ih{\mathrm{\parallel }}_B\le r<{\rho }_h$. Therefore $e^{-\mathrm{\Phi }(ih)-L_{h}r}\mu ({\bar{B}}_B(ih,r))\le {\int }_{{\bar{B}}_B(ih,r)}{e}^{-\mathrm{\Phi }(u)}\mu (du)\le e^{-\mathrm{\Phi }(ih)+L_{h}r}\mu ({\bar{B}}_B(ih,r))\mathrm{.}$ For fixed $h_1,h_2\in E$, divide the two integral bounds and cancel the common normalizer $Z$ in [F1]. The factors from the Lipschitz errors tend to $1$, and [F7] gives $\frac{\mathrm{\Pi }({\bar{B}}_B(i{h}_1,r))}{\mathrm{\Pi }({\bar{B}}_B(i{h}_2,r))}⟶e^{\mathrm{\Phi }(i{h}_2)-\mathrm{\Phi }(i{h}_1)}{e}^{(\mathrm{\parallel }{h}_2{\mathrm{\parallel }}_E^2-\mathrm{\parallel }{h}_1{\mathrm{\parallel }}_E^2)\mathrm{/}2}=e^{I(i{h}_2)-I(i{h}_1)}\mathrm{.}$ The denominator is positive for small $r$: [F4] gives positive prior mass to every open ball about $i{h}_2$ (since $i{h}_2\in iE$), and the likelihood is strictly positive and continuous there.

Choose a minimizing sequence $(h_n)$ in $E$ and, by [F9], pass to a subsequence for which $\mathrm{\parallel }{h}_n{\mathrm{\parallel }}_E\to L$. Its images lie in a fixed multiple of the closed $E$-unit ball, whose image is compact in $B$ by [F6]. Pass to a further subsequence with $i{h}_n\to v$ in $B$; compactness means $v=ih$ for some $h\in E$. For each $\mathrm{\ell }\in B^{\ast }$, [F5] gives $\mathrm{\mid }\mathrm{\ell }(ih)\mathrm{\mid }={\lim }_n\mathrm{\mid }\mathrm{\ell }(i{h}_n)\mathrm{\mid }={\lim }_n\mathrm{\mid }⟨h_n,[\mathrm{\ell }]{⟩}_E\mathrm{\mid }\le L\mathrm{\parallel }[\mathrm{\ell }]{\mathrm{\parallel }}_E\mathrm{.}$ The classes $[\mathrm{\ell }]$ are dense in $E$ by [F5]. For every unit $g\in E$, choose nonzero classes $[{\mathrm{\ell }}_n]\to g$ and normalize them; then $[{\mathrm{\ell }}_n]\mathrm{/}\mathrm{\parallel }[{\mathrm{\ell }}_n]{\mathrm{\parallel }}_E\to g$. Thus the normalized covariance classes are dense in the unit sphere, and the supremum of $\mathrm{\mid }⟨h,[\mathrm{\ell }]{⟩}_E\mathrm{\mid }$ over them is $\mathrm{\parallel }h{\mathrm{\parallel }}_E$. The displayed bound gives $\mathrm{\parallel }h{\mathrm{\parallel }}_E\le L$. Local Lipschitz continuity gives $\mathrm{\Phi }(i{h}_n)\to \mathrm{\Phi }(ih)$, and therefore $I(ih)\le {\lim }_n(\mathrm{\Phi }(i{h}_n)+\frac{1}{2}\mathrm{\parallel }{h}_n{\mathrm{\parallel }}_E^2)={\inf }_{E}I\mathrm{.}$ Thus $I$ attains its finite minimum.

By [F7], the posterior ball ratio for fixed $ih,ik\in iE$ is $e^{I(ik)-I(ih)}$. This limit is at least $1$ for every $k\in E$ exactly when $I(ih)\le I(ik)$ for every $k\in E$. By [F8], those points are precisely the Cameron–Martin ratio modes. The comparison is only over fixed centers in $iE$; no supremum over all $B$ centers has been controlled, so it does not establish a global strong small-ball MAP.