Hellinger stability and posterior discretization

By [F1], choose an at most countable dense set $C\subseteq X$. Since $\mu (X)=1$ and $\mu (\mathrm{\varnothing })=0$ by [F7], $X$ is nonempty, so $C$ is nonempty. Let ${\mathbb{Q}}_{>0}$ be the positive rationals. By [F3], $C\times {\mathbb{Q}}_{>0}$ is a nonempty at most countable set, so [F4] gives a surjection $e:\mathbb{N}\to C\times {\mathbb{Q}}_{>0}$. The composite of $e$ with $(c,q)\mapsto B(c,q)$ is a surjection onto ${\mathcal{B}}_0:=\{B(c,q):c\in C,q\in {\mathbb{Q}}_{>0}\}$, so ${\mathcal{B}}_0$ is at most countable. Each of its members is open by [F2]. It is a base: if $x\in U$ with $U$ open, choose $r>0$ so $B(x,r)\subseteq U$, choose $c\in C$ with $\mathrm{\parallel }x-c{\mathrm{\parallel }}_X<r\mathrm{/}3$, and choose rational $q$ with $\mathrm{\parallel }x-c{\mathrm{\parallel }}_X<q<r-\mathrm{\parallel }x-c{\mathrm{\parallel }}_X$ using density in [F3]. Then $x\in B(c,q)$, and for every $z\in B(c,q)$ the triangle inequality gives $\mathrm{\parallel }z-x{\mathrm{\parallel }}_X<q+\mathrm{\parallel }x-c{\mathrm{\parallel }}_X<r$, so $x\in B(c,q)\subseteq U$.

Let $\mathcal{N}:=\{B\in {\mathcal{B}}_0:\mu (B)=0\}$ and $N:=\bigcup \mathcal{N}$. By [F3], $\mathcal{N}$ is at most countable; by [F5] $N$ is Borel, and by [F6] its measure is zero (with the empty-family case giving $N=\mathrm{\varnothing }$). If $x\in N$, a zero-measure open basis ball contains $x$, so $x\mathrm{\notin }\mathrm{supp}\mu$. If $x\mathrm{\notin }N$, every open neighborhood $U$ of $x$ contains some $B\in {\mathcal{B}}_0$ with $x\in B\subseteq U$ by step 1.1; since $B\mathrm{\notin }\mathcal{N}$, $\mu (B)>0$, and [F6] gives $\mu (U)\ge \mu (B)>0$. Thus $X\setminus N=\mathrm{supp}\mu$, which is Borel and has measure one.

The fixed-radius sections $s\mapsto M_i(R,s)$ are Borel and nondecreasing by [F8] and the hypotheses. Since $M_2\ge 0$, $e^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)}\le e^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)}(1+M_2(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X{)}^2)$, so [F7] gives ${\int }_D{e}^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)}d{\mu }_0\le J_R<\mathrm{\infty }$. Thus the hypotheses of [F2] hold for every $y\in B_Y(0,R)$, and all stated pointwise updates are probability measures with $0<Z(y)<\mathrm{\infty }$.

By the Statement choose a bounded Borel set $E\subseteq D$ with ${\mu }_0(E)>0$, and choose $K<\mathrm{\infty }$ with $\mathrm{\parallel }u{\mathrm{\parallel }}_X\le K$ for all $u\in E$. By [F3], the support has full measure; applying [F12] to $E$ gives ${\mu }_0(E\cap \mathrm{supp}{\mu }_0)={\mu }_0(E)>0$, so choose $u_{\ast }\in E\cap \mathrm{supp}{\mu }_0$.

Fix $u\in D$ and $y,y^{\mathrm{\prime }}\in B_Y(0,R)$, and write $a=\mathrm{\Phi }(u;y)$, $b=\mathrm{\Phi }(u;y^{\mathrm{\prime }})$, and $\delta =\mathrm{\parallel }y-y^{\mathrm{\prime }}{\mathrm{\parallel }}_Y$. Both $a,b\ge -M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)$, so every real between them has the same lower bound. By [F6] and the data-Lipschitz hypothesis, $\mathrm{\mid }{e}^{-a\mathrm{/}2}-e^{-b\mathrm{/}2}\mathrm{\mid }\le \frac{1}{2}{e}^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)\mathrm{/}2}\mathrm{\mid }a-b\mathrm{\mid }\le \frac{1}{2}{e}^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)\mathrm{/}2}{M}_2(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)\delta \mathrm{.}$ Squaring and integrating, using [F7], gives $\mathrm{\parallel }{e}^{-\mathrm{\Phi }(\cdot ;y)\mathrm{/}2}-e^{-\mathrm{\Phi }(\cdot ;y^{\mathrm{\prime }})\mathrm{/}2}{\mathrm{\parallel }}_{L^2({\mu }_0)}\le \frac{1}{2}\sqrt{J_R}\delta$ by the definition of the $L^2$ norm [F11].

By [F6], the same lower envelope bounds the derivative of $t\mapsto e^{-t}$ between $a$ and $b$ by $e^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)}$. Hence $\mathrm{\mid }Z(y)-Z(y^{\mathrm{\prime }})\mathrm{\mid }\le \delta {\int }_D{e}^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)}{M}_2(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)d{\mu }_0\le J_R\delta ,$ where $M_2\le 1+M_2^2$ and [F7] was used in the last inequality. For the first inequality, monotonicity and additivity apply to the pointwise bounds $e^{-\mathrm{\Phi }(u;y)}\le e^{-\mathrm{\Phi }(u;y^{\mathrm{\prime }})}+\mathrm{\mid }{e}^{-\mathrm{\Phi }(u;y)}-e^{-\mathrm{\Phi }(u;y^{\mathrm{\prime }})}\mathrm{\mid }$ and its reverse.

Continuity of $u\mapsto \mathrm{\Phi }(u;0)$ at $u_{\ast }$ gives $\rho >0$ such that $\mathrm{\Phi }(u;0)\le \mathrm{\Phi }(u_{\ast };0)+1$ for $u\in D\cap B_X(u_{\ast },\rho )$. The support property gives ${\mu }_0(B_X(u_{\ast },\rho ))>0$, and [F12] with ${\mu }_0(D)=1$ gives $m_{\ast }:={\mu }_0(D\cap B_X(u_{\ast },\rho ))={\mu }_0(B_X(u_{\ast },\rho ))>0$. Every such $u$ satisfies $\mathrm{\parallel }u{\mathrm{\parallel }}_X<K+\rho$. For any $y\in B_Y(0,R)$, the data-Lipschitz bound between $y$ and $0$, followed by monotonicity of $M_2$ in its norm argument, gives $\mathrm{\Phi }(u;y)\le \mathrm{\Phi }(u_{\ast };0)+1+M_2(R,K+\rho )R\mathrm{.}$ By monotonicity and positive homogeneity of the nonnegative integral [F7], this yields $Z(y)\ge m_{\ast }\exp [-\mathrm{\Phi }(u_{\ast };0)-1-M_2(R,K+\rho )R]=:z_R>0$, uniformly on the data ball.

The lower envelope gives $e^{-\mathrm{\Phi }(u;y)}\le e^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)}$, and [F7] therefore yields $Z(y)\le {\int }_D{e}^{M_1(R,\mathrm{\parallel }u{\mathrm{\parallel }}_X)}d{\mu }_0\le J_R$. Together with step 2.1 this proves the asserted uniform normalizer bounds.

Put $a_y=e^{-\mathrm{\Phi }(\cdot ;y)\mathrm{/}2}$ and $q_y=a_y\mathrm{/}\sqrt{Z(y)}$, so [F4] gives $h({\mathrm{\Pi }}^y,{\mathrm{\Pi }}^{y^{\mathrm{\prime }}})=\mathrm{\parallel }{q}_y-q_{y^{\mathrm{\prime }}}{\mathrm{\parallel }}_2$. Add and subtract $a_{y^{\mathrm{\prime }}}\mathrm{/}\sqrt{Z(y)}$ and use the triangle inequality in $L^2$ to obtain $\mathrm{\parallel }{q}_y-q_{y^{\mathrm{\prime }}}{\mathrm{\parallel }}_2\le \frac{\mathrm{\parallel }{a}_y-a_{y^{\mathrm{\prime }}}{\mathrm{\parallel }}_2}{\sqrt{Z(y)}}+\frac{\mathrm{\mid }\sqrt{Z(y^{\mathrm{\prime }})}-\sqrt{Z(y)}\mathrm{\mid }}{\sqrt{Z(y)}}\mathrm{.}$ The first term is at most $\sqrt{J_R}\delta \mathrm{/}(2\sqrt{z_R})$ by steps 2.1 and 1.3. Since $\mathrm{\parallel }{a}_{y^{\mathrm{\prime }}}{\mathrm{\parallel }}_2^2={\int }_D{e}^{-\mathrm{\Phi }(u;y^{\mathrm{\prime }})}d{\mu }_0=Z(y^{\mathrm{\prime }})$, the second term equals $\mathrm{\mid }Z(y^{\mathrm{\prime }})-Z(y)\mathrm{\mid }\mathrm{/}[\sqrt{Z(y)}(\sqrt{Z(y^{\mathrm{\prime }})}+\sqrt{Z(y)})]$, hence is at most $J_R\delta \mathrm{/}(2z_R)$ by steps 2.1 and 1.4. This proves the stated $C_R$ bound.

Under the separate assumption [A1], apply [F5] to the two explicit posterior densities relative to the common probability measure ${\mu }_0$. Thus $\mathrm{\parallel }{\mathrm{\Pi }}^y-{\mathrm{\Pi }}^{y^{\mathrm{\prime }}}{\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}\le h({\mathrm{\Pi }}^y,{\mathrm{\Pi }}^{y^{\mathrm{\prime }}})$, and step 3.2 gives the optional TV estimate without adding an axiom to the Hellinger estimate.

Since $M_2\ge 0$, $e^{M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_X)}\le e^{M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_X)}(1+M_2(\mathrm{\parallel }u{\mathrm{\parallel }}_X{)}^2)$, so [F6] gives ${\int }_D{e}^{M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_X)}d{\mu }_0\le J<\mathrm{\infty }$. Apply [F1] to $\mathrm{\Phi }$ and, separately for each fixed $N$, to ${\mathrm{\Phi }}_N$; this gives $0<Z<\mathrm{\infty }$ and $0<Z_N<\mathrm{\infty }$ for every $N$, and defines the displayed probability updates.

Fix $u\in D$ and $N$. Both potential values are at least $-M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_X)$, so every real between them has that lower bound. By [F5] and the potential-error assumption, $\mathrm{\mid }{e}^{-{\mathrm{\Phi }}_N(u)}-e^{-\mathrm{\Phi }(u)}\mathrm{\mid }\le e^{M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_X)}{M}_2(\mathrm{\parallel }u{\mathrm{\parallel }}_X){\psi }_N\mathrm{.}$ Integrating and using $M_2\le 1+M_2^2$ gives $\mathrm{\mid }{Z}_N-Z\mathrm{\mid }\le J{\psi }_N\mathrm{.}$ The absolute difference of the integrals is bounded by the integral of the absolute difference: apply monotonicity and additivity to $e^{-{\mathrm{\Phi }}_N}\le e^{-\mathrm{\Phi }}+\mathrm{\mid }{e}^{-{\mathrm{\Phi }}_N}-e^{-\mathrm{\Phi }}\mathrm{\mid }$ and its reverse. Since $Z>0$ and ${\psi }_N\to 0$, there is $N_0$ such that $J{\psi }_N\le Z\mathrm{/}2$ for every $N\ge N_0$, and then $Z_N\ge Z\mathrm{/}2$.

The same mean-value argument for $t\mapsto e^{-t\mathrm{/}2}$ gives $\mathrm{\mid }{e}^{-{\mathrm{\Phi }}_N(u)\mathrm{/}2}-e^{-\mathrm{\Phi }(u)\mathrm{/}2}\mathrm{\mid }\le \frac{1}{2}{e}^{M_1(\mathrm{\parallel }u{\mathrm{\parallel }}_X)\mathrm{/}2}{M}_2(\mathrm{\parallel }u{\mathrm{\parallel }}_X){\psi }_N\mathrm{.}$ Squaring and integrating yields $\mathrm{\parallel }{e}^{-{\mathrm{\Phi }}_N\mathrm{/}2}-e^{-\mathrm{\Phi }\mathrm{/}2}{\mathrm{\parallel }}_{L^2({\mu }_0)}\le \frac{1}{2}\sqrt{J}{\psi }_N\mathrm{.}$

Put $a=e^{-\mathrm{\Phi }\mathrm{/}2}$ and $a_N=e^{-{\mathrm{\Phi }}_N\mathrm{/}2}$. The square roots of the posterior densities are $q=a\mathrm{/}\sqrt{Z}$ and $q_N=a_N\mathrm{/}\sqrt{Z_N}$ by [F2]. Add and subtract $a_N\mathrm{/}\sqrt{Z}$ and use the triangle inequality in $L^2$ to obtain $\mathrm{\parallel }q-q_N{\mathrm{\parallel }}_2\le \frac{\mathrm{\parallel }a-a_N{\mathrm{\parallel }}_2}{\sqrt{Z}}+\frac{\mathrm{\mid }\sqrt{Z_N}-\sqrt{Z}\mathrm{\mid }}{\sqrt{Z}}\mathrm{.}$ By [F3], the left side is $h(\mathrm{\Pi },{\mathrm{\Pi }}_N)$. Also $\mathrm{\parallel }{a}_N{\mathrm{\parallel }}_2^2={\int }_D{e}^{-{\mathrm{\Phi }}_N}d{\mu }_0=Z_N$, which gives the displayed second term after applying [F10]. The first term is at most $\sqrt{J}{\psi }_N\mathrm{/}(2\sqrt{Z})$ by step 2.2. The second equals $\mathrm{\mid }{Z}_N-Z\mathrm{\mid }\mathrm{/}[\sqrt{Z}(\sqrt{Z_N}+\sqrt{Z})]$, so by step 2.1 and $Z_N\ge Z\mathrm{/}2$ it is at most $J{\psi }_N\mathrm{/}Z$. Therefore $h(\mathrm{\Pi },{\mathrm{\Pi }}_N)\le (\frac{\sqrt{J}}{2\sqrt{Z}}+\frac{J}{Z}){\psi }_N,$ which is the claimed bound with a finite constant independent of $N\ge N_0$.

Under the separate assumption [A1], apply [F4] to the two explicit posterior densities relative to ${\mu }_0$. This gives $\mathrm{\parallel }\mathrm{\Pi }-{\mathrm{\Pi }}_N{\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}\le h(\mathrm{\Pi },{\mathrm{\Pi }}_N)$, and step 3.1 gives the optional TV consequence without adding an axiom to the Hellinger estimate.