Finite-dimensional small-ball MAP equals the density mode

By [F1], the prior and noise have positive continuous Gaussian densities proportional to $\exp (-\frac{1}{2}{u}^T{C}^{-1}u)$ and $\exp (-\frac{1}{2}(y-Au{)}^T{\mathrm{\Gamma }}^{-1}(y-Au))$, respectively. For any $w$, $w^T{\mathrm{\Gamma }}^{-1}w\ge 0$: write $w=\mathrm{\Gamma }v$, so $w^T{\mathrm{\Gamma }}^{-1}w=v^T\mathrm{\Gamma }v\ge 0$. Thus ${\mathrm{\Phi }}_y\ge 0$ and $e^{-{\mathrm{\Phi }}_y}\le 1$, so the evidence is finite by [F6] and the fact that the prior is a probability. Write $f_0$ for the prior density. The function $f_0$ is positive and continuous at $0$, and $e^{-{\mathrm{\Phi }}_y}$ is also positive and continuous there. On a sufficiently small open ball $O$ about $0$, there are constants $c_0,c_1>0$ with $f_0\ge c_0$ and $e^{-{\mathrm{\Phi }}_y}\ge c_1$. By [F5], $\lambda (O)>0$; [F6] then gives $N(0,C)(O)={\int }_O{f}_{0}d\lambda \ge c_0\lambda (O)>0$ and $Z\ge c_{1}N(0,C)(O)>0$. The posterior update [F1] is therefore defined. Its Lebesgue density is proportional to the product of the prior density and likelihood, hence to $e^{-I_y(u)}$. Put $Q=C^{-1}+A^T{\mathrm{\Gamma }}^{-1}A$. For $v\mathrm{\ne }0$, $v^{T}Qv=v^T{C}^{-1}v+(Av{)}^T{\mathrm{\Gamma }}^{-1}(Av)>0$: if $z=C^{-1}v$, then $v^T{C}^{-1}v=z^{T}Cz>0$, and the second term is nonnegative by the preceding argument. Thus $Q$ is positive definite. Completing the square gives the unique minimizer $m=Q^{-1}{A}^T{\mathrm{\Gamma }}^{-1}y$, and the posterior is the nondegenerate Gaussian $N(m,Q^{-1})$.

For every $r>0$, apply [F2] to this posterior and the loss $L_r(x)={\mathbf{1}}_{\{\mathrm{\parallel }x{\mathrm{\parallel }}_2>r\}}$. Its sublevel sets are the closed radius-$r$ ball and the whole space, so it is bounded, symmetric and quasi-convex. Thus every ball of radius $r$ has posterior mass at most the ball centered at $m$, and the supremum in [F3] is attained at $m$. Consequently $m$ is a strong small-ball MAP.

Let $q$ be the positive continuous posterior density. By [F5], every closed Euclidean ball has positive finite Lebesgue measure; the inclusions $B(z,r)\subseteq \bar{B}(z,r)\subseteq B(z,2r)$ and the open-ball result in [F5] give the volume bounds, while [F4] makes the volumes of same-radius balls equal. For any fixed $z\in {\mathbb{R}}^d$, the absolute difference between $\lambda (\bar{B}(z,r){)}^{-1}\mathrm{\Pi }(\bar{B}(z,r))$ and $q(z)$ is at most ${\sup }_{\mathrm{\parallel }x-z{\mathrm{\parallel }}_2\le r}\mathrm{\mid }q(x)-q(z)\mathrm{\mid }$, by [F6]; this tends to zero by continuity. The same holds at any $v$, and the common ball volume cancels; hence $\mathrm{\Pi }(\bar{B}(z,r))\mathrm{/}\mathrm{\Pi }(\bar{B}(v,r))\to q(z)\mathrm{/}q(v)$. Since $q$ is proportional to $e^{-I_y}$, the finite-dimensional small-ball ratio agrees with the density and Onsager–Machlup comparison.