Posterior Prediction, Credible Sets, and Bayes Actions

Fix $y\in \mathsf{Y}$ and regard ${\mathrm{\Pi }}_y$ as a probability measure on $\mathrm{\Theta }$. By [F2], $Q$ is a probability kernel, so [F3] gives a joint probability measure $J_y$ on $\mathrm{\Theta }\times \mathsf{Z}$ satisfying $J_y(A\times B)={\int }_A{Q}_{\theta }(B){\mathrm{\Pi }}_y(d\theta )$ and the iterated-integral identity for every nonnegative product-measurable function. Taking $A=\mathrm{\Theta }$ shows $Q_y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}(B)=J_y(\mathrm{\Theta }\times B)$; hence it is the second marginal of a probability measure and is itself a probability measure. The same identity applied to $(\theta ,z)\mapsto g(z)$ gives the displayed integral formula.

Fix $B\in \mathcal{Z}$. The function $\theta \mapsto Q_{\theta }(B)$ is $\mathcal{T}$-measurable by [F2], so $(y,\theta )\mapsto Q_{\theta }(B)$ is $\mathcal{Y}\otimes \mathcal{T}$-measurable as the composition with the measurable coordinate projection. By [F4], $y\mapsto {\int }_{\mathrm{\Theta }}{Q}_{\theta }(B){\mathrm{\Pi }}_y(d\theta )$ is measurable. By [F1] this is $Q_y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}(B)$, proving kernel measurability.

Let $s_2(\theta )=\int z^2{Q}_{\theta }(dz)$. It is measurable by [F4]. Since $\int s_{2}d{\mathrm{\Pi }}_y<\mathrm{\infty }$, the set $E_y=\{s_2<\mathrm{\infty }\}$ is measurable and has ${\mathrm{\Pi }}_y(E_y)=1$. On $E_y$, $\mathrm{\mid }z\mathrm{\mid }\le 1+z^2$ gives a finite first absolute moment, so $\mu (\theta )$ is finite and measurable by [F4]. Expanding the square in the variance definition [F3] gives $v(\theta )=s_2(\theta )-\mu (\theta {)}^2\ge 0$, hence $\mu (\theta {)}^2\le s_2(\theta )$; therefore $\mu$ is square-integrable under ${\mathrm{\Pi }}_y$ and $v$ is integrable.

For the joint-model hypotheses in the Statement, let $B$ be a measurable future event and $G\in \mathcal{G}$. The function $\theta \mapsto Q_{\theta }(B)$ is measurable by [F9]. Apply [F5] to the posterior conditional law and this bounded function to obtain ${\int }_G{Q}_Y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}(B)dP={\int }_G{Q}_X(B)dP$. Since $G\in \mathcal{G}\subseteq \mathcal{H}$, apply [F5] again to the conditional law $Q_X$ to get ${\int }_G{Q}_X(B)dP=P(G\cap \{Z\in B\})$. By [F2], the mixture has probability-measure sections and measurable evaluations; these event identities therefore make $Q_Y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}$ a regular conditional distribution of $Z$ given $\mathcal{G}$ under [F6].

By [F1], $Q_y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}$ is the mixture defined from $Q_{\theta }$ and ${\mathrm{\Pi }}_y$. Apply [F2] to $g(z)=z^2$ to obtain $\int z^2{Q}_y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}(dz)=\int s_{2}d{\mathrm{\Pi }}_y<\mathrm{\infty }$. Apply [F2] to $\mathrm{\mid }z\mathrm{\mid }$ and use $\mathrm{\mid }z\mathrm{\mid }\le 1+z^2$ to see that $Q_y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}$ has finite first absolute moment. Applying [F2] to the nonnegative positive and negative parts of $z$ and subtracting their finite integrals gives $\int z{Q}_y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}(dz)=\int \mu d{\mathrm{\Pi }}_y$.

Apply [F5] to $z$ and $z^2$ under $Q_X$. Writing $m:=\mu (X)$ and $s:=s_2(X)$, their conditional-integration identities give $\mathbb{E}[1_{A}Z]=\mathbb{E}[1_{A}m]$ and $\mathbb{E}[1_A{Z}^2]=\mathbb{E}[1_{A}s]$ for every $A\in \mathcal{H}$, and $\mathbb{E}[s]=\mathbb{E}[Z^2]<\mathrm{\infty }$. Applying [F5] to $s_2$ under ${\mathrm{\Pi }}_Y$ gives a conditional version of $s$ with expectation $\mathbb{E}[s]$; hence $D:=\{y:\int s_2(\theta ){\mathrm{\Pi }}_y(d\theta )<\mathrm{\infty }\}$ has full probability. Take the zero-filled representative $\bar{s}(Y):=\int s_2(\theta ){\mathrm{\Pi }}_y(d\theta )$ on $D$ and zero elsewhere. Then $\mathbb{E}[\bar{s}(Y)]=\mathbb{E}[s]=\mathbb{E}[Z^2]<\mathrm{\infty }$ and ${\mathrm{\Pi }}_Y(E)=1$ almost surely for $E=\{\theta :s_2(\theta )<\mathrm{\infty }\}$. The conditioning identity for $X$ then gives $P(X\in E)=1$. By step 1.1, $m^2\le s$ and $0\le v(X)\le s$ almost surely; hence $m\in L^2$ and $v(X)$ is integrable, with $v(X)=s-m^2$ almost surely. Because $X$ is $\mathcal{H}$-measurable and these moment functions are measurable, $m,s,v(X)$ are $\mathcal{H}$-measurable. The first event identity extends from bounded simple $\mathcal{H}$-measurable multipliers to $1_{A}m$ by simple approximation and truncation; $2\mathrm{\mid }mZ\mathrm{\mid }\le m^2+Z^2$ and $m^2\in L^1$ justify the limits. Thus $\mathbb{E}[1_{A}mZ]=\mathbb{E}[1_A{m}^2]$, and expansion gives $\mathbb{E}[1_A(Z-m{)}^2]=\mathbb{E}[1_A(s-m^2)]=\mathbb{E}[1_{A}v(X)]$ for every $A\in \mathcal{H}$. This proves eventwise that $m$ is the conditional-mean version of $Z$ and $v(X)$ is the conditional-variance version given $\mathcal{H}$.

On $E_y$, $s_2=v+{\mu }^2$, and off $E_y$ the set is ${\mathrm{\Pi }}_y$-null. By [F3] and step 2.1, ${\mathrm{Var}}_{Q_y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}}(Z)=\int s_{2}d{\mathrm{\Pi }}_y-(\int \mu d{\mathrm{\Pi }}_y{)}^2=\int vd{\mathrm{\Pi }}_y+\int {\mu }^{2}d{\mathrm{\Pi }}_y-(\int \mu d{\mathrm{\Pi }}_y{)}^2=\int vd{\mathrm{\Pi }}_y+{\mathrm{Var}}_{{\mathrm{\Pi }}_y}(\mu )$. This proves both displayed kernel identities without choosing conditional-expectation representatives.

On the full-measure data domain where $b(Y):=\int s_{2}d{\mathrm{\Pi }}_Y<\mathrm{\infty }$, set $c(Y):=\int \mu d{\mathrm{\Pi }}_Y$, $d(Y):=\int {\mu }^{2}d{\mathrm{\Pi }}_Y$, and $e(Y):=\int vd{\mathrm{\Pi }}_Y$, and extend $b,c,d,e$ by zero on its null complement, using the signed zero-filled extension for $c$. By [F5] and the kernel identities in steps 2.1 and 3.1, $c,b$ are conditional first- and second-moment versions of $Z$ given $\mathcal{G}$, while $c,d,e$ are conditional moments of $m,m^2,v(X)$ given $\mathcal{G}$. Since $Y$ is $\mathcal{G}$-measurable, these functions of $Y$ are $\mathcal{G}$-measurable. Also $\mathbb{E}[b(Y)]=\mathbb{E}[Z^2]<\mathrm{\infty }$, $c^2\le b$ almost surely since $b-c^2={\mathrm{Var}}_{Q_Y^{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}}}(Z)\ge 0$, and $d\le b<\mathrm{\infty }$ almost surely since ${\mu }^2\le s_2$.

For each $A\in \mathcal{G}$, the same simple-multiplier and truncation argument as in step 2.2, now applied to $m$ with center $c(Y)$ and to $Z$ with that same center, gives $\mathbb{E}[1_A(m-c(Y){)}^2]=\mathbb{E}[1_A(d-c(Y{)}^2)]$ and $\mathbb{E}[1_A(Z-c(Y){)}^2]=\mathbb{E}[1_A(b(Y)-c(Y{)}^2)]$; the bounds $2\mathrm{\mid }cm\mathrm{\mid }\le c^2+m^2$ and $2\mathrm{\mid }cZ\mathrm{\mid }\le c^2+Z^2$ justify the respective truncations. Thus $d-c^2$ and $b-c^2$ are the conditional-variance versions of $m$ and $Z$ given $\mathcal{G}$. The kernel identity in step 3.1 gives $b-c^2=e+(d-c^2)$, so the conditional variance of $Z$ given $\mathcal{G}$ is the expected conditional variance given $\mathcal{H}$ plus the conditional variance of the conditional mean. These event-identity versions are choice-free; under [A1] they may also be read as conditional-expectation equivalence classes by [F5], while [F8] gives the eventwise version convention. No value on a null data fiber is thereby made canonical.

Fix $y$. By [F8, F10], the sets $(-\mathrm{\infty },n]$ increase to $\mathbb{R}$, so [F2] gives $F_y(n)\uparrow 1$ and some natural $n$ satisfies $F_y(n)\ge u$. By [F4] choose a rational $r>n$; then [F7] gives $F_y(r)\ge u$, so $S_y=\{r\in \mathbb{Q}:F_y(r)\ge u\}$ is nonempty. Again by [F8, F10], the sets $(-\mathrm{\infty },-n]$ decrease to the empty set; [F3] gives some natural $m$ with $F_y(-m)<u$. Choose a rational $s<-m$ by [F4]. For any rational $r\le s$, [F7] gives $F_y(r)\le F_y(-m)<u$, so every member of $S_y$ exceeds $s$ and $S_y$ is bounded below. Hence [F6, F10] gives its real infimum $q_u(y)$.

For each $c\in \mathbb{R}$, the infimum definition and nonemptiness give $\{y:q_u(y)<c\}={\bigcup }_{\begin{array}{c}r\in \mathbb{Q}\\ r<c\end{array}}\{y:F_y(r)\ge u\}\mathrm{.}$ Each set on the right is measurable by [F1], and the union is countable by [F5], so $q_u$ is measurable.

Suppose ${\mathrm{\Pi }}_y((-\mathrm{\infty },q_u(y)])<u$. By [F9], the sets $(-\mathrm{\infty },q_u(y)+1\mathrm{/}n]$ decrease to $(-\mathrm{\infty },q_u(y)]$; they have finite measure by [F1], so [F3] gives an $n$ with $F_y(q_u(y)+1\mathrm{/}n)<u$. By [F4], choose rational $r$ with $q_u(y)<r<q_u(y)+1\mathrm{/}n$. Then [F7] gives $F_y(r)\le F_y(q_u(y)+1\mathrm{/}n)<u$. Consequently no member of $S_y$ is at most $r$, so $r$ is a lower bound of $S_y$. Then $q_u(y)=\inf S_y\ge r>q_u(y)$, a contradiction; therefore $u\le {\mathrm{\Pi }}_y((-\mathrm{\infty },q_u(y)])$.

Suppose ${\mathrm{\Pi }}_y((-\mathrm{\infty },q_u(y)))>u$. By [F9], the sets $(-\mathrm{\infty },q_u(y)-1\mathrm{/}n]$ increase to $(-\mathrm{\infty },q_u(y))$, so [F2] gives an $n$ with $F_y(q_u(y)-1\mathrm{/}n)>u$. By [F4], choose rational $r$ with $q_u(y)-1\mathrm{/}n<r<q_u(y)$. Then [F7] gives $F_y(r)\ge F_y(q_u(y)-1\mathrm{/}n)>u$, so $r\in S_y$, contradicting that $q_u(y)$ is a lower bound of $S_y$. Thus ${\mathrm{\Pi }}_y((-\mathrm{\infty },q_u(y)))\le u$. Combining with step 2.2 proves the CDF bracket.

Since $\alpha \mathrm{/}2<1-\alpha \mathrm{/}2$, the defining rational sets for the two quantiles are nested in the direction that gives $q_{\alpha \mathrm{/}2}(y)\le q_{1-\alpha \mathrm{/}2}(y)$. Both endpoint functions are measurable by [F2].

Put $l=q_{\alpha \mathrm{/}2}(y)$ and $u=q_{1-\alpha \mathrm{/}2}(y)$. The disjoint difference $(-\mathrm{\infty },u]\setminus (-\mathrm{\infty },l)$ is $[l,u]$, so ${\mathrm{\Pi }}_y([l,u])={\mathrm{\Pi }}_y((-\mathrm{\infty },u])-{\mathrm{\Pi }}_y((-\mathrm{\infty },l))\mathrm{.}$ By [F2], the first term is at least $1-\alpha \mathrm{/}2$ and the second is at most $\alpha \mathrm{/}2$, giving posterior mass at least $1-\alpha$.

The complement of the interval membership graph consists of pairs with $x<q_{\alpha \mathrm{/}2}(y)$ or $x>q_{1-\alpha \mathrm{/}2}(y)$. By [F4], the first relation is the countable union over $r\in \mathbb{Q}$ of $\{y:q_{\alpha \mathrm{/}2}(y)>r\}\times (-\mathrm{\infty },r),$ and the second is the countable union over $r\in \mathbb{Q}$ of $\{y:q_{1-\alpha \mathrm{/}2}(y)<r\}\times (r,\mathrm{\infty })\mathrm{.}$ Each is a measurable rectangle by endpoint measurability, so [F3] makes the graph measurable.

The graph measurability from step 2.1 and the mass bound from step 1.2 satisfy [F1], so $C_{\alpha }$ is a posterior credible set.

By [F2], $m(y)=q_{1\mathrm{/}2}(y)$ is measurable and ${\mathrm{\Pi }}_y((-\mathrm{\infty },m(y)])\ge 1\mathrm{/}2$.

Since ${\mathrm{\Pi }}_y$ is a probability measure, ${\mathrm{\Pi }}_y([m(y),\mathrm{\infty }))=1-{\mathrm{\Pi }}_y((-\mathrm{\infty },m(y)))\ge 1\mathrm{/}2$ by [F2]. Thus both inequalities in [F1] hold, so $m(y)$ is a median for every $y$.

The squared loss $L(\theta ,a)=(\theta -a{)}^2$ is continuous on ${\mathbb{R}}^2$, hence product-Borel measurable as required by [F3]. Let $s_2(y)=\int {\theta }^2{\mathrm{\Pi }}_y(d\theta )$. The function $(y,\theta )\mapsto {\theta }^2$ is product-measurable, so [F4] makes $s_2$ measurable and $D_2=\{s_2<\mathrm{\infty }\}$ measurable. On $D_2$, $2\mathrm{\mid }\theta \mathrm{\mid }\le 1+{\theta }^2$ gives a finite absolute first moment; [F4] therefore makes the signed mean $\mu$ measurable on $D_2$.

Fix $y\in D_2$ and use $(\mathbb{R},\mathcal{B}(\mathbb{R}),{\mathrm{\Pi }}_y)$ as a probability space with trivial sub-sigma-algebra $\mathcal{G}=\{\mathrm{\varnothing },\mathbb{R}\}$. The identity variable $\theta$ is a real random variable by [F5] and is square-integrable. The constant function $\mu (y)$ is $\mathcal{G}$-measurable and satisfies ${\int }_H\theta d{\mathrm{\Pi }}_y={\int }_H\mu (y)d{\mathrm{\Pi }}_y$ for both possible $H\in \mathcal{G}$; by [F2] it is a conditional-expectation version.

By [F1] under [A1], this conditional expectation uniquely minimizes squared distance over $L^2(\mathcal{G})$. The real-valued $\mathcal{G}$-measurable functions are exactly constants, and every finite constant is in $L^2({\mathrm{\Pi }}_y)$; hence $\mu (y)$ uniquely minimizes $\int (\theta -a{)}^2{\mathrm{\Pi }}_y(d\theta )$ over all real actions. This risk is finite because both $\theta$ and the constant action are square-integrable.

By step 1.1 the mean rule is measurable on $D_2$, and by step 2.1 it has finite risk and attains the minimum on every fiber. The definition [F3] therefore makes it a Bayes action rule on $D_2$.

The absolute loss $L(\theta ,a)=\mathrm{\mid }\theta -a\mathrm{\mid }$ is continuous on ${\mathbb{R}}^2$, hence product-Borel measurable as required by [F2]. The function $(y,\theta )\mapsto \mathrm{\mid }\theta \mathrm{\mid }$ is product-measurable, so [F4] makes its posterior integral measurable and hence $D_1$ a measurable data domain. For $y\in D_1$ and every finite $a$, $\mathrm{\mid }\theta -a\mathrm{\mid }\le \mathrm{\mid }\theta \mathrm{\mid }+\mathrm{\mid }a\mathrm{\mid }$ makes the posterior risk finite.

Fix $y\in D_1$ and a median $m$. For any $a>m$, put $d=a-m$ and partition $\mathbb{R}$ into $A=(-\mathrm{\infty },m]$, $B=(m,a)$, and $C=[a,\mathrm{\infty })$. On these three sets respectively, $\mathrm{\mid }\theta -a\mathrm{\mid }-\mathrm{\mid }\theta -m\mathrm{\mid }$ equals $d$, $a+m-2\theta$, and $-d$. On $B$, $a+m-2\theta \ge -d$. Therefore ${\rho }_y(a)-{\rho }_y(m)\ge d\{{\mathrm{\Pi }}_y(A)-{\mathrm{\Pi }}_y(B)-{\mathrm{\Pi }}_y(C)\}=d\{2{\mathrm{\Pi }}_y(A)-1\}\ge 0$, where the last inequality is the first median condition [F1].

If $a<m$, put $d=m-a$ and partition into $A=(-\mathrm{\infty },a]$, $B=(a,m)$, and $C=[m,\mathrm{\infty })$. On these sets the loss difference is respectively $-d$, $2\theta -a-m$, and $d$; on $B$, $2\theta -a-m\ge -d$. Hence ${\rho }_y(a)-{\rho }_y(m)\ge d\{-{\mathrm{\Pi }}_y(A)-{\mathrm{\Pi }}_y(B)+{\mathrm{\Pi }}_y(C)\}=d\{2{\mathrm{\Pi }}_y(C)-1\}\ge 0$ by the second median condition [F1]. Equality holds when $a=m$. Thus every median is a pointwise minimizer; [F3] supplies a measurable median rule, whose finite risk on $D_1$ and attained minimum satisfy [F2].

Since the parameter and action spaces are finite with all subsets measurable, the zero–one loss is product-measurable as required by [F1]. For $j=1,\dots ,k$, put $p_j(y)={\mathrm{\Pi }}_y(\{j\})$. By [F2], every $p_j$ is measurable. The zero–one posterior risk of action $j$ is ${\rho }_y(j)={\sum }_{\mathrm{\ell }\mathrm{\ne }j}{p}_{\mathrm{\ell }}(y)=1-p_j(y)\mathrm{.}$ Thus the risk is minimized exactly by maximizing $p_j(y)$.

Since $k\ge 1$ is finite, the nonempty finite list $p_1(y),\dots ,p_k(y)$ has a maximum. Define $j^{\ast }(y)$ to be the least index attaining it. For each $j$, $\{y:j^{\ast }(y)=j\}={\bigcap }_{\mathrm{\ell }>j}\{y:p_j(y)\ge p_{\mathrm{\ell }}(y)\}\cap {\bigcap }_{\mathrm{\ell }<j}\{y:p_j(y)>p_{\mathrm{\ell }}(y)\}\mathrm{.}$ Both intersections are finite, and each set is measurable because the $p_{\mathrm{\ell }}$ are measurable; hence $j^{\ast }$ is measurable. Its risk is finite and it attains the pointwise minimum from step 1.1, so [F1] makes it a Bayes action rule. Choosing the least maximizing index uses only the displayed finite order, not AC.