Pointwise continuity alone does not guarantee the zero-tolerance limit

Equip ${\mathbb{N}}_{\ge 1}$ with the discrete metric $d(n,m)=0$ for $n=m$ and $1$ otherwise. A Cauchy sequence, meaning that its tail terms are pairwise less than every positive radius apart, is eventually constant by taking radius $1\mathrm{/}2$, so this metric is complete. The set is countable and dense in itself; each singleton is open, so every subset is open and its Borel sigma-algebra is the power set. The identity is therefore a Borel isomorphism to this Polish presentation, and [F1] gives standard Borelness.

Put ${\pi }_n=1\mathrm{/}[n(n+1)]$. The partial sums telescope: ${\sum }_{n=1}^N{\pi }_n={\sum }_{n=1}^N(\frac{1}{n}-\frac{1}{n+1})=1-\frac{1}{N+1}⟶1$ by [F4]. Therefore $\mathrm{\Pi }={\sum }_{n\ge 1}{\pi }_n{\delta }_n$ is a probability measure by [F3].

Define $b(s)=\frac{1}{4}{\mathbf{1}}_{[-1,1]}(s)$, $w_n=1\mathrm{/}(100n^3)$, ${\varphi }_n(s)=(4w_n{)}^{-1}{\mathbf{1}}_{[1\mathrm{/}n-w_n,1\mathrm{/}n+w_n]}(s)$, and $m_n=b+{\varphi }_n$. The two summands are nonnegative measurable functions with integrals $1\mathrm{/}2$ each, so $m_n$ is a probability density. Let $K(n,ds)=m_n(s)ds$. It is a probability kernel: every section is a probability measure, and every function of $n$ is measurable on the discrete parameter space. For measurable $A\subseteq {\mathbb{N}}_{\ge 1}$ and Lebesgue-measurable $C\subseteq \mathbb{R}$, the ABC definition gives $M(A\times C)={\sum }_{n\in A}{\pi }_n{\int }_C{m}_n(s)ds$. To extend this identity from rectangles, let $E\subseteq {\mathbb{N}}_{\ge 1}\times \mathbb{R}$ be product-measurable and put $E_n=\{s:(n,s)\in E\}$. The section map $s\mapsto (n,s)$ is measurable, so each $E_n$ is Lebesgue-measurable; because ${\mathbb{N}}_{\ge 1}$ is countable discrete, $E={⨆}_{n\ge 1}(\{n\}\times E_n)$. Countable additivity and the rectangle identity give $M(E)={\sum }_{n\ge 1}{\pi }_n{\int }_{E_n}{m}_n(s){\lambda }_1(ds)\mathrm{.}$ The function $m(n,s):=m_n(s)$ is product-measurable because ${\mathbb{N}}_{\ge 1}$ is countable discrete and every $m_n$ is measurable. The prior $\mathrm{\Pi }$ is sigma-finite and Lebesgue measure is sigma-finite, so [F7] applied to ${\mathbf{1}}_{E}m$ gives ${\int }_{E}m(n,s)(\mathrm{\Pi }\times {\lambda }_1)(d(n,s))={\sum }_{n\ge 1}{\pi }_n{\int }_{E_n}{m}_n(s){\lambda }_1(ds)\mathrm{.}$ Thus $M$ has the specified jointly measurable density on the full product sigma-algebra, not only on rectangles. The identity summary is measurable.

For each fixed $n$, the spike interval is a positive distance from $0$, since $1\mathrm{/}n-w_n>0$. On a sufficiently small neighborhood of $0$, $m_n(s)=1\mathrm{/}4$; hence $m_n$ is continuous at $0$ and $m_n(0)=1\mathrm{/}4$. Thus $z_0={\sum }_n{\pi }_n{m}_n(0)=1\mathrm{/}4$, and the density-selected zero-tolerance posterior is $\mathrm{\Pi }$ itself. However, for every $\delta >0$, all sufficiently large $n$ have $1\mathrm{/}n<\delta$, and on the neighborhood $(-\delta ,\delta )$ any common pointwise envelope $H(n)$ must satisfy $H(n)\ge m_n(1\mathrm{/}n)=\frac{1}{4}+25n^3\mathrm{.}$ Since ${\pi }_n\ge 1\mathrm{/}(2n^2)$, ${\sum }_n{\pi }_{n}H(n)\ge \frac{25}{2}{\sum }_{n}n=\mathrm{\infty }$. Every prior atom has positive mass, so an almost-everywhere envelope condition also forces this bound for every $n$. No integrable local envelope exists.

For integer $q\ge 2$, take ${ϵ}_q=1\mathrm{/}q$ and $B_q=[-1\mathrm{/}q,1\mathrm{/}q]$. The base density contributes $1\mathrm{/}(2q)$ to every parameter's ball probability by the box-volume formula [F5]. If $n<q$, then $1\mathrm{/}n-w_n>1\mathrm{/}q$: indeed $(q-n)\mathrm{/}(nq)\ge 1\mathrm{/}(n(n+1))>1\mathrm{/}(100n^3)$. If $n>q$, then $1\mathrm{/}n+w_n<1\mathrm{/}q$: here $(n-q)\mathrm{/}(qn)\ge 1\mathrm{/}(q(q+1))>1\mathrm{/}(100(q+1{)}^3)\ge 1\mathrm{/}(100n^3)$. Thus spikes for $n<q$ contribute zero; the $n=q$ ball cuts its symmetric spike at the center and captures half its mass, namely $1\mathrm{/}4$; and spikes for $n>q$ lie wholly in the ball and contribute $1\mathrm{/}2$.

By [F6] and the telescoping tail ${\sum }_{n>q}{\pi }_n=1\mathrm{/}(q+1)$, the acceptance probability is $p_q=\frac{1}{2q}+\frac{1}{4q(q+1)}+\frac{1}{2(q+1)},q{p}_q⟶1.$ For the fixed parameter event $A=\{1\}$, its unnormalized accepted mass is ${\pi }_1\mathrm{/}(2q)=1\mathrm{/}(4q)$, because its spike lies outside $B_q$. Hence ${\mathrm{\Pi }}_{{ϵ}_q}(A\mid 0)=(1\mathrm{/}(4q))\mathrm{/}{p}_q\to 1\mathrm{/}4$, whereas the density-selected limit assigns $A$ mass $\mathrm{\Pi }(A)=1\mathrm{/}2$. The total-variation distance is at least this event-probability difference, which tends to $1\mathrm{/}4$. Thus the claimed convergence fails without the integrable-envelope condition.