Likelihood potentials and posterior propriety — Examples and Counterexamples

The set $D=U=\{0,1\}$ is Borel and nonempty. The function ${\mu }_0(A)=\frac{1}{2}{\mathbf{1}}_{\{0\in A\}}+\frac{1}{2}{\mathbf{1}}_{\{1\in A\}}$ is countably additive: in a disjoint sequence of Borel sets each of the two points occurs at most once, so the sum of assigned masses equals the mass of the union. Also ${\mu }_0(\mathbb{R})=1$ and ${\mu }_0(D)=1$, making it a prior on the stated domain by [F8]. The identity $F:D\to \mathbb{R}$ is measurable because every subset of finite $D$ is Borel, and it satisfies the forward-map interface [F9].

For any $u\in D$ and $y\in \mathbb{R}$, put $a:=2\pi$; since $\log a$ occurs in the real-valued potential, [F15] gives $a>0$. Put $b:=\frac{1}{2}(y-u{)}^2$ and $c:=\frac{1}{2}\log a$. By [F15], $\exp (2c)=a$. By [F16], $\exp (c{)}^2=\exp (2c)=a$; by [F18], $\exp (c)>0$. Thus uniqueness of the nonnegative square root [F17] gives $\exp (c)=\sqrt{2\pi }$, and [F18] gives $\exp (-c)=1\mathrm{/}\sqrt{2\pi }$. Applying [F16] again, $e^{-\mathrm{\Phi }(u;y)}=\exp (-(b+c))=\exp (-b)\exp (-c)=\frac{1}{\sqrt{2\pi }}\exp \text{\hspace{0.17em}⁣}(-\frac{(y-u{)}^2}{2})=L(u;y)$. The last expression is the displayed Gaussian density, so the likelihood and potential agree pointwise.

By [A1] and [F1], each $N(u,1)$ is a probability law with the displayed density; since $D$ is finite, every evaluation $u\mapsto N(u,1)(B)$ is measurable, so [F10] gives a probability kernel. By [F2] and [F3], Lebesgue measure is sigma-finite. The potential and likelihood are jointly Borel: for each of the two $u$ values their data sections are continuous, each singleton $\{u\}$ is Borel in $D$, and each preimage of a Borel set is a finite union of rectangles $\{u\}\times B_u$ in $\mathcal{B}(D)\otimes \mathcal{B}(\mathbb{R})$ by [F13]. Thus [F4] gives a proper Bayesian model with the stated prior and sampling kernel.

The finite subspace $D=\{0,1\}$ is discrete: the open balls of radius $1\mathrm{/}2$ about $0$ and $1$ isolate the two points, so every real-valued function on $D$ is continuous. For every $u\in D$ and $y\in \mathbb{R}$, $\mathrm{\Phi }(u;y)$ is finite. By [F11], $(y-u{)}^2\ge 0$ and $a\ge -\mathrm{\mid }a\mathrm{\mid }$ for $a:=\frac{1}{2}\log (2\pi )$, so $\mathrm{\Phi }(u;y)\ge -C$ where $C:=\mathrm{\mid }a\mathrm{\mid }\ge 0$. The constant envelope $M_1(r)=C$ is Borel, and [F12] gives ${\int }_D{e}^{M_1(\mathrm{\parallel }u\mathrm{\parallel })}{\mu }_0(du)=e^C{\mu }_0(D)=e^C<\mathrm{\infty }$. Applying [F5] with ${\mathsf{Y}}_0=\mathbb{R}$ proves $0<Z(y)<\mathrm{\infty }$ and propriety of the normalized update for every datum.

By [F14], integrate the likelihood section against the prior. Step 1.2 gives $L(u;y)=e^{-\mathrm{\Phi }(u;y)}$ for both atoms, so this likelihood evidence equals $Z_{\mathrm{\Phi }}(y)$; using the two prior atoms gives $Z(y)=\frac{1}{2}L(0;y)+\frac{1}{2}L(1;y)$. Step 3.1 gives $0<Z(y)<\mathrm{\infty }$. Since $D$ is finite, the likelihood section is simple, so [F12] gives the numerator for any $A\subseteq D$ as $\frac{1}{2}L(0;y){\mathbf{1}}_{\{0\in A\}}+\frac{1}{2}L(1;y){\mathbf{1}}_{\{1\in A\}}$. Dividing by $Z(y)$ yields the stated posterior weights.

Apply [F6] with parameter $u\in D$, prior ${\mu }_0$, dominating measure Lebesgue, and the jointly Borel density $L(u;y)$. Its evidence is exactly the $Z(y)$ computed in step 4.1, so the displayed ${\mathrm{\Pi }}^y$ is a data-indexed posterior-kernel version. By [F7], changing its values on a data-marginal-null set leaves the conditional identities unchanged.

The half-half set function ${\mu }_0$ is a probability: in any disjoint Borel sequence each of $0$ and $1$ occurs in at most one member, so the mass of the union equals the sum of the masses, and ${\mu }_0(\mathbb{R})=1$. Since ${\mu }_0(D)=1$, it is a prior on $D$ by [F1]. The constant map $F(u)=0$ is a forward map on the nonempty truth set $U=D$ by [F12], and its restriction is measurable because every preimage is $D$ or $\mathrm{\varnothing }$. For each $u\in D$, $Q(u,\cdot )={\delta }_0$ is a probability measure by [F4], and every evaluation $u\mapsto Q(u,B)$ is measurable on finite $D$; hence [F7] makes $Q$ a probability kernel. Thus [F3] gives a proper Bayesian model.

With counting measure on $\mathsf{Y}$, the density is $L(u,y)={\mathbf{1}}_{\{y=0\}}$ for both parameter values; its sum over $y\in \{0,1\}$ is one, so it represents the stated kernel. The potential and density are jointly measurable: each singleton rectangle $\{u\}\times \{y\}$ is measurable, so every subset of finite $D\times \mathsf{Y}$ is a finite union of measurable rectangles by [F13]. Integrating the kernel against the half-half prior gives data law concentrated at $0$, and the same density gives $Z(0)=\frac{1}{2}+\frac{1}{2}=1$ and $Z(1)=0$. At $y=1$, [F9] rules out any finite positive scaling of the zero likelihood measure as a probability. The potential equals $+\mathrm{\infty }$ on $D$ at this datum, so the finite-section hypothesis in [F11] fails.

The explicit dominated kernel from [F8] uses the normalized update at $y=0$ and prior filler ${\mu }_0$ at $y=1$. The data law gives $\{1\}$ probability zero by step 2.1, so [F10] allows replacing that filler by ${\delta }_1$ without changing the conditional identities. Both fillers are probability measures on $D$ by [F4], but neither is obtained by normalizing the zero-evidence fiber.

For each $m\in \mathbb{N}$, the initial set $E_m=\{0,1,\dots ,m\}$ has counting measure $m+1<\mathrm{\infty }$ by [F1] and [F2], and ${\bigcup }_m{E}_m=\mathbb{N}$. Thus $\mu$ is sigma-finite by [F3]. By [F16], $\mathbb{N}$ is infinite, so [F1] gives $\mu (\mathbb{N})=+\mathrm{\infty }$.

The counting-integral formula [F6] gives $Z(0)={\int }_{\mathbb{N}}\frac{e^{-n^2\mathrm{/}2}}{\sqrt{2\pi }}\mu (dn)=\frac{1}{\sqrt{2\pi }}{\sum }_{n=0}^{\mathrm{\infty }}{e}^{-n^2\mathrm{/}2}$. Its $n=0$ term is $1\mathrm{/}\sqrt{2\pi }>0$ by [F8].

AC implies countable choice by [F5], so the normal-density interface [F4] applies: for each $n$, $N(n,1)$ is a probability law with the displayed Gaussian density. Since the parameter sigma-algebra is $\mathcal{P}(\mathbb{N})$, every evaluation $n\mapsto N(n,1)(B)$ is measurable, and the likelihood section $n\mapsto L_y(n)$ is measurable by [F15]; thus the sampling family satisfies [F13]. The map $F(n)=n$ on the nonempty admissible set $U=\mathbb{N}$ satisfies the forward-map interface [F14]. By [F11] this is not a proper Bayesian model with prior $\mu$, because [step 1.1] shows the prior mass is infinite.

For every $n\ge 1$, $n^2-n=n(n-1)\ge 0$, since $n$ and $n-1$ are nonnegative; hence $n^2\ge n$. Monotonicity of the exponential [F8] gives $e^{-n^2\mathrm{/}2}\le e^{-n\mathrm{/}2}$. Let $r=e^{-1\mathrm{/}2}$. By [F8], $0<r<1$, and [F7] makes ${\sum }_{n=1}^{\mathrm{\infty }}{r}^n$ finite. Comparison of nonnegative partial sums now bounds the evidence series in step 1.2 by $1+{\sum }_{n=1}^{\mathrm{\infty }}{r}^n<\mathrm{\infty }$. Hence $0<Z(0)<\mathrm{\infty }$.

Applying [F10] to $L_0$ and the sigma-finite measure $\mu$ gives the probability with singleton masses $L_0(n)\mathrm{/}Z(0)=e^{-n^2\mathrm{/}2}\mathrm{/}{\sum }_j{e}^{-j^2\mathrm{/}2}$, as stated. Its fixed-data meaning and lack of a prior-predictive probability follow from [F9]; [F11] confirms that no proper Bayesian model is supplied by this prior. The function-space propriety theorem [F12] does not apply because its prior is a probability, whereas [step 1.1] gives $\mu (\mathbb{N})=+\mathrm{\infty }$.