Hierarchical Bayes and partial pooling — Examples

The prior is sigma-finite: the sets $E_n=[1\mathrm{/}n,n]$ increase to $(0,\mathrm{\infty })$ and $\mathrm{\Pi }(E_n)={\int }_{1\mathrm{/}n}^{n}dt\mathrm{/}t=2\log n<\mathrm{\infty }$. Integrating out $\theta$ gives the evidence at fixed $t$, $m(0\mid t)={\int }_{\mathbb{R}}\frac{e^{-{\theta }^2\mathrm{/}(2t)}}{\sqrt{2\pi t}}\frac{e^{-{\theta }^2\mathrm{/}2}}{\sqrt{2\pi }}d\theta =\frac{1}{\sqrt{2\pi (1+t)}}$. Indeed, the exponent is $-(1+t^{-1}){\theta }^2\mathrm{/}2$; the substitution $z=\sqrt{1+t^{-1}}\theta$ reduces the integral to the unit normal integral [F2]. It is positive and finite for every $t>0$.

The evidence-weighted density of $t$ is $q(t)=1\mathrm{/}(t\sqrt{2\pi (1+t)})$ by [A2] and step 1.1. For each $k\ge 0$, let $I_k=(2^{-(k+1)},2^{-k})$. On $I_k\subset (0,1)$, $q(t)\ge 1\mathrm{/}(t\sqrt{4\pi })$ and $1\mathrm{/}t\ge 2^k$, while the interval has length $2^{-(k+1)}$. Hence ${\int }_{I_k}q(t)dt\ge 1\mathrm{/}(2\sqrt{4\pi })$, uniformly in $k$. The intervals are disjoint, so summing their lower bounds shows ${\int }_0^{1}q(t)dt=\mathrm{\infty }$. By [F3], no finite positive scaling makes this likelihood-weighted measure a posterior probability.

The specified model supplies the prior and sampling kernels [F1]; their densities are given by [F4]. For fixed $t>0$, multiply the two densities and integrate over $\theta$. The integrand is $(2\pi \sqrt{t}{)}^{-1}\exp (-(1+t^{-1}){\theta }^2\mathrm{/}2)$. Substituting $z=\sqrt{1+t^{-1}}\theta$ and using [F5] for the unit normal integral gives the marginal evidence $m_t(0)=1\mathrm{/}\sqrt{2\pi (1+t)}$, which is positive and finite for $t=1,4$.

The two prior masses are equal, so [F2] gives posterior odds $P(t=1\mid y=0)\mathrm{/}P(t=4\mid y=0)=m_1(0)\mathrm{/}{m}_4(0)=\sqrt{5\mathrm{/}2}>1$. Thus the posterior mode is $t=1$, and the posterior weights $w=P(t=1\mid y=0)$ and $1-w$ are both strictly between zero and one.

By [F3], $\theta \mid t=1,y=0\sim N(0,1\mathrm{/}2)$ and $\theta \mid t=4,y=0\sim N(0,4\mathrm{/}5)$. The full posterior is their mixture with weights $w$ and $1-w$, so its second moment is $w\mathrm{/}2+(1-w)4\mathrm{/}5>1\mathrm{/}2$.

Plugging in the posterior mode $t=1$ gives only $N(0,1\mathrm{/}2)$, whose second moment is $1\mathrm{/}2$. This differs from the full posterior's second moment in step 3.1, so the plug-in posterior is not the full posterior mixture.

Set $a_j={\tau }^2\mathrm{/}({\tau }^2+s_j^2)$. Algebra in [F1] gives ${\widehat{\theta }}_j=a_j{\bar{y}}_j+(1-a_j)\mu$ and $V_j={\tau }^2{s}_j^2\mathrm{/}({\tau }^2+s_j^2)$. Since both variances are positive, $0<a_j<1$, so for finite positive ${\tau }^2$ the group mean uses both its own data and the population mean.

As ${\tau }^2\downarrow 0$, $a_j\to 0$, $V_j\to 0$, and ${\widehat{\theta }}_j\to \mu$. At ${\tau }^2=0$, [F2] makes the conditional prior ${\delta }_{\mu }$, so every group parameter equals $\mu$ and conditioning on the data leaves it there. This is complete pooling.

As ${\tau }^2\to \mathrm{\infty }$, $a_j\to 1$, $V_j\to s_j^2$, and ${\widehat{\theta }}_j\to {\bar{y}}_j$. These are the group-only posterior mean and variance, so they describe the no-pooling limit. No posterior at ${\tau }^2=\mathrm{\infty }$ is asserted.