Hierarchical Bayes and partial pooling

The product kernel $K$ is a probability kernel by [F1]. Construct the finite product measure recursively: the one-factor case is ${\nu }_0$, and each next product is sigma-finite by [F2] applied to the preceding sigma-finite product and the next ${\nu }_j$. By [F7], each coordinate pullback $(s,t)\mapsto f_j(s,t_j)$ is measurable on $S\times {\prod }_{j<n}{T}_j$. For two nonnegative extended-measurable functions $u,v$ and $a\ge 0$, $\{uv>a\}={\bigcup }_{q\in \mathbb{Q},q>0}\{u>q\}\cap \{v>a\mathrm{/}q\};$ when $uv>a$, one can choose a positive rational $q$ with $a\mathrm{/}v<q<u$, with the evident choice of any $0<q<u$ when $a=0$; this also covers infinite factors and excludes a zero factor under the stated $0\cdot \mathrm{\infty }=0$ convention. For $a<0$ the superlevel set is the whole domain. Thus finite multiplication preserves measurability by [F6], and $f$ is jointly measurable.

Fix $s$ and define ${\lambda }_s(A)={\int }_{A}f(s,t)\nu (dt)$. By [F4], ${\lambda }_s$ is a measure. Repeated Tonelli [F3] and the probability-kernel density identities give ${\lambda }_s\text{\hspace{0.17em}⁣}\left({\prod }_{j<n}{T}_j\right)=\int {\prod }_{j<n}{f}_j(s,t_j)d\nu ={\prod }_{j<n}\int f_j(s,t_j){\nu }_j(d{t}_j)={\prod }_{j<n}{K}_j(s,T_j)=1.$ Hence ${\lambda }_s$ is a probability measure.

For a measurable rectangle $R={\prod }_{j<n}{B}_j$, repeated Tonelli and the density identities yield ${\lambda }_s(R)={\prod }_{j<n}{\int }_{B_j}{f}_j(s,t_j){\nu }_j(d{t}_j)={\prod }_{j<n}{K}_j(s,B_j)=K(s,R),$ where the last equality is [F1]. Rectangles form a generating pi-system containing the whole product. Both measures are probabilities by step 1.2 and [F1], so the constant exhaustion by the whole product has common finite mass one. [F5] therefore gives ${\lambda }_s=K(s,\cdot )$ on the full product sigma-algebra. This holds for every $s$, proving the density assertion.

Put $\mathrm{\Xi }=H\times {\mathrm{\Theta }}^m$ with its finite product sigma-algebra. The model definition supplies the proper latent prior $P_0$ on $\mathrm{\Xi }$ and the probability kernel from $\mathrm{\Xi }$ to the product observation space $\mathcal{Y}$; apply [F2] to obtain the joint probability law $P_0(d\xi )K(\xi ,dy)$ on $\mathrm{\Xi }\times \mathcal{Y}$. Let $\mathrm{\Omega }=\mathrm{\Xi }\times \mathcal{Y}$ with this law and let $\xi$ and $Y$ be the coordinate projections. By [F5], both projections are measurable.

The target $\mathrm{\Xi }$ is standard Borel by [F3], since $H$ and $\mathrm{\Theta }$ are standard Borel and there are finitely many factors. The coordinate map $\xi :\mathrm{\Omega }\to \mathrm{\Xi }$ is a measurable random element, and $\sigma (Y)$ is a sub-sigma-algebra of the canonical product sigma-algebra. By [F4] under [A1], $\xi$ has a regular conditional distribution given $\sigma (Y)$. The definition of a regular conditional distribution requires every section to be a probability measure, including at exceptional sample points. The theorem asserts no normalization formula on a fixed observation fiber.

By [F1] and [F3], the latent prior $P_0$ is a probability measure and the model kernel gives a joint probability law for the latent tuple and the observations. By [F2], that observation kernel is dominated by the sigma-finite product measure $\nu$ with the specified jointly measurable density $L(\xi ,y)={\prod }_j{f}_j({\theta }_j,y_j)$. Thus all hypotheses of [F4] apply with prior $P_0$ and likelihood density $L$.

The resulting evidence is $m(y)={\int }_{\mathrm{\Xi }}{L}_y(\xi )P_0(d\xi )=Z(y)$. On every fiber with $0<Z(y)<\mathrm{\infty }$, [F4] gives the probability measure $P_y(A)=Z(y{)}^{-1}{\int }_A{L}_{y}d{P}_0$ and the conditional-event identities. On fibers with zero or infinite evidence [F4] uses the probability filler $P_0$, so its kernel remains a regular conditional distribution there without dividing by zero or infinity. The same theorem says these filled fibers have prior-predictive measure zero.

Fix a group $j$ and specialize [F5] to one observation: identify its parameter with ${\theta }_j$, set ${\mu }_0=\mu$, ${\tau }_0^2={\tau }^2$, $n=1$, $y_1={\bar{y}}_j$, and ${\sigma }^2=s_j^2$. The model and pointwise density specification in [F1] supply this normal prior and the canonical normal likelihood density used by [F5], while [A1] and [A2] verify the choice and positive-variance hypotheses of [F5]. The theorem gives a finite positive group evidence and posterior variance $V_j=(1\mathrm{/}{\tau }^2+1\mathrm{/}{s}_j^2{)}^{-1}$ and mean ${\widehat{\theta }}_j=V_j(\mu \mathrm{/}{\tau }^2+{\bar{y}}_j\mathrm{/}{s}_j^2)$, as stated.

For each group put ${\mu }_j(A)={\int }_A{f}_j(\theta ,{\bar{y}}_j)N(\mu ,{\tau }^2)(d\theta )$ and let $m_j={\mu }_j(\mathbb{R})>0$. By [F5] and step 1.1, ${\mu }_j\mathrm{/}{m}_j=Q_j=N({\widehat{\theta }}_j,V_j)$. Apply [F3] to the hierarchy with the proper point-mass hyperprior at the fixed $(\mu ,{\tau }^2)$. Its latent prior on ${\mathbb{R}}^m$ is the product of the group priors by [F1], and its likelihood density is ${\prod }_j{f}_j({\theta }_j,{\bar{y}}_j)$ by [F2]. Repeated Tonelli [F4] shows that the normalized posterior assigns each measurable rectangle ${\prod }_j{A}_j$ mass ${\prod }_j{\mu }_j(A_j)\mathrm{/}{\prod }_j{m}_j={\prod }_j{Q}_j(A_j)$, the rectangle mass of ${⨂}_j{Q}_j$ by [F9]. By [F8], rectangles form a generating pi-system. Both are probability measures agreeing on it; the constant exhaustion by the full space has common finite mass one, so [F6] makes them equal. This proves conditional independence and the displayed formula for every group.

Now suppose $(\mu ,{\tau }^2)$ is random with a proper hyperprior supported on $H$. For fixed observed summaries, put $M_j((\mu ,{\tau }^2),A)={\int }_A{f}_j(\theta ,{\bar{y}}_j)N(\mu ,{\tau }^2)(d\theta )$. The factor $f_j(\cdot ,{\bar{y}}_j)$ is measurable by the section fact [F8]; composing it with the parameter-coordinate projection shows that $(\eta ,\theta )\mapsto 1_A(\theta )f_j(\theta ,{\bar{y}}_j)$ is product-measurable. Since $G_{\eta }=N(\mu ,{\tau }^2)$ is a probability kernel, [F7] makes $M_j(\cdot ,A)$ measurable. Hence $m_j=M_j(\cdot ,\mathbb{R})$ and $Q_j(A)=M_j(\cdot ,A)\mathrm{/}{m}_j$ are measurable, with $m_j>0$ by step 1.1. Repeated Tonelli [F4] and the full-posterior formula [F3] give, for measurable $C\subseteq H$ and $A_1,\dots ,A_m\subseteq \mathbb{R}$, $P_y(C\times {\prod }_j{A}_j)=Z(y{)}^{-1}{\int }_C{\prod }_j{M}_j((\mu ,{\tau }^2),A_j)\mathrm{\Pi }(d\mu ,d{\tau }^2)$. Taking every $A_j=\mathbb{R}$ shows ${\mathrm{\Pi }}_y(C)=Z(y{)}^{-1}{\int }_C{\prod }_j{m}_{j}d\mathrm{\Pi }$. Taking only $A_j=A$ and all other sets equal to $\mathbb{R}$ therefore gives $P_y({\theta }_j\in A)={\int }_H{Q}_j(A){\mathrm{\Pi }}_y(d\mu ,d{\tau }^2)$. By [F5] and step 1.1, $Q_j=N({\widehat{\theta }}_j,V_j)$, so the group posterior is its conditional normal law averaged over the full hyperparameter posterior. If ${\mathrm{\Pi }}_y$ is a point mass, this mixture equals its single conditional normal component; in general it need not equal a plug-in law.

Fix a current group $j$ and $y\in \mathcal{Y}$. For each measurable future-event set $A$, the formula $Q_{j,y}^{\mathrm{c}\mathrm{u}\mathrm{r}}(A)=\int K_j({\theta }_j,A)P_y(d\xi )$ gives zero on $A=\mathrm{\varnothing }$ and one on the full observation space by [F1] and [F2]. If $(A_k{)}_{k\ge 0}$ are disjoint, then $K_j({\theta }_j,{\bigcup }_k{A}_k)={\lim }_N{\sum }_{k\le N}{K}_j({\theta }_j,A_k)$; [F4] passes this increasing limit through the $P_y$ integral and gives countable additivity. The integrand is measurable in $\xi$ by the kernel definition and coordinate projection, so [F3] makes $y\mapsto Q_{j,y}^{\mathrm{c}\mathrm{u}\mathrm{r}}(A)$ measurable against the posterior kernel $y\mapsto P_y$. Thus each $Q_j^{\mathrm{c}\mathrm{u}\mathrm{r}}$ is a probability kernel from $\mathcal{Y}$ to ${\mathcal{Y}}_j$.

For a new group, define $R_{\eta }(A)=\int K_{\mathrm{n}\mathrm{e}\mathrm{w}}(\theta ,A)G_{\eta }(d\theta )$. The empty-set and whole-space evaluations are zero and one. For disjoint $(A_k)$, countable additivity of each $K_{\mathrm{n}\mathrm{e}\mathrm{w}}(\theta ,\cdot )$ and [F4] pass the increasing partial sums through the $G_{\eta }$ integral; hence $R_{\eta }$ is a probability measure. [F3] makes $\eta \mapsto R_{\eta }(A)$ measurable, so $R$ is a probability kernel. Mixing $R_{\eta }(A)$ over $P_y$ through the measurable coordinate $\eta$ and repeating the countable-additivity argument shows that $Q_y^{\mathrm{n}\mathrm{e}\mathrm{w}}(A)$ is a probability measure and [F3] makes its evaluations measurable in $y$. Thus $Q^{\mathrm{n}\mathrm{e}\mathrm{w}}$ is a probability kernel from $\mathcal{Y}$ to $\tilde{\mathcal{Y}}$. Its integral through the marginal $P_y^{\eta }$ is the same iterated-kernel law.

For either prediction let $H(\xi ,A)$ be its conditional future-event probability given the latent tuple: $K_j({\theta }_j,A)$ for a current group, or $R_{\eta }(A)$ for a new group. Steps 1.1–1.2 make $H$ a probability kernel. In the stipulated extension, [F7] ensures that the product of the observation kernel and this future kernel is a probability kernel from $\mathrm{\Xi }$ to the product of data and future-observation spaces; applying [F6] with prior $P_0$ gives a joint probability law on $(\xi ,Y,\tilde{Y})$. Because every section of $H$ has total mass one, marginalizing out $\tilde{Y}$ leaves the original joint law of $(\xi ,Y)$, so [F1] still supplies its posterior conditional law $P_y$. For data event $B$ and future event $A$, the triple law gives $P(Y\in B,\tilde{Y}\in A)=E[1_{\{Y\in B\}}H(\xi ,A)]$. Applying the event identity in [F5] to $H(\xi ,A)$ turns the right side into ${\int }_B\int H(\xi ,A)P_y(d\xi )P_Y(dy)={\int }_B{Q}_y(A)P_Y(dy)$. This verifies the conditional-event identity for every data event $B$ and proves that both predictive kernels are the conditional laws of the future observations. Since $P_y=P_0$ off ${\mathcal{Y}}_{+}$, their exceptional-fiber predictions are the prior-predictive laws.