Marginal likelihoods, Bayes factors, and empirical Bayes — Examples

Both group means and the grand mean are zero, so $MSB=2(0^2+0^2)=0$. Each of the four within-group residuals is either $-1$ or $1$, hence $MSW=(1+1+1+1)\mathrm{/}2=2$. By [F1], the raw estimate is $(0-2)\mathrm{/}2=-1$.

The map from the four observations to $(MSB-MSW)\mathrm{/}2$ is a polynomial divided by the fixed positive integer $2$, hence continuous. At the displayed table its value is $-1$ by step 1.1, so some open neighborhood of the table still has negative estimates. The latent vector $(a_1,a_2,{ϵ}_{11},{ϵ}_{12},{ϵ}_{21},{ϵ}_{22})=(0,0,-1,1,-1,1)$ maps to that table. Since every coordinate has a strictly positive normal density by [F2], an open rectangle around this latent vector has positive probability by independence [F3]; it can be chosen inside the preimage of the negative-estimate neighborhood. Thus negative estimates occur with positive probability even when ${\tau }^2>0$, so replacing them by zero changes the estimator and does not prove a structural zero.

By [F2], $m_0(0)=(2\pi {\sigma }^2{)}^{-1\mathrm{/}2}$. The alternative evidence is $m_1(0)={\int }_{\mathbb{R}}\frac{e^{-{\theta }^2\mathrm{/}(2{\tau }^2)}}{\sqrt{2\pi {\tau }^2}}\frac{e^{-{\theta }^2\mathrm{/}(2{\sigma }^2)}}{\sqrt{2\pi {\sigma }^2}}d\theta$. The exponent coefficient is $({\tau }^2+{\sigma }^2)\mathrm{/}(2{\tau }^2{\sigma }^2)$; substituting $z=\theta \sqrt{({\tau }^2+{\sigma }^2)\mathrm{/}({\tau }^2{\sigma }^2)}$ and using the standard normal integral gives $m_1(0)=(2\pi ({\tau }^2+{\sigma }^2){)}^{-1\mathrm{/}2}$. Both evidences are finite and positive.

By [F1], the alternative-to-null Bayes factor is $B{F}_{10}(0)=m_1(0)\mathrm{/}{m}_0(0)=\sqrt{{\sigma }^2\mathrm{/}({\sigma }^2+{\tau }^2)}$. As ${\tau }^2\to \mathrm{\infty }$ with ${\sigma }^2$ fixed, this tends to zero. Therefore the evidence ratio changes with the proper alternative prior's spread even though the observed datum and point-null model remain fixed.

By [A1, F3, F4], $\lambda$ is sigma-finite, so ${\mathrm{\Pi }}_c=c\lambda$ is sigma-finite. By [F5], ${\mathrm{\Pi }}_c([-n,n])=2cn$ for every positive integer $n$, hence ${\mathrm{\Pi }}_c(\mathbb{R})=+\mathrm{\infty }$ and it is improper in the sense of [F2]. By [F6], its formal likelihood integral is ${\int }_{\mathbb{R}}\frac{c}{\sqrt{2\pi {\sigma }^2}}\exp \text{\hspace{0.17em}⁣}(-\frac{(y-\theta {)}^2}{2{\sigma }^2})d\theta$. The substitution $z=(y-\theta )\mathrm{/}\sigma$ has absolute Jacobian $\sigma$, so this equals $c{\int }_{\mathbb{R}}(2\pi {)}^{-1\mathrm{/}2}{e}^{-z^2\mathrm{/}2}dz=c$. This value is positive and finite for every fixed $c>0$, but ${\mathrm{\Pi }}_c(\mathbb{R})=+\mathrm{\infty }$, so it is not an evidence under [F1].

For the two copies of the same normal-location likelihood specified above, with model-specific constants $c_1,c_2>0$, the formal ratio is $c_2\mathrm{/}{c}_1$ by step 1.1. Rescaling either improper prior changes this ratio arbitrarily while leaving its unnormalized density proportional to Lebesgue measure; hence no generic Bayes factor is defined by these improper priors.

The finite discrete spaces are standard Borel by [F4]. The two-point hyperprior is proper, $G_0={\delta }_0$ and $G_1={\delta }_1$ are probability kernels, and each singleton observation measure ${\nu }_j$ is finite, hence sigma-finite. Its observation kernel has the specified density $f_j({\theta }_j,\ast )=1$, jointly measurable on the finite product space. Thus the hierarchy in [F1] is well-defined and its joint evidence is $Z(\ast )=1$. By [F2], the full posterior equals the prior: it assigns probability $1\mathrm{/}2$ to $(\eta ,{\theta }_1,{\theta }_2)=(0,0,0)$ and $1\mathrm{/}2$ to $(1,1,1)$.

For either fixed $\eta$, the conditional marginal likelihood is $m(\ast \mid \eta )=1$, so both values attain the maximum. The map from the singleton data space to $\eta =0$ is measurable, hence [F3] permits the plug-in choice $\widehat{\eta }(\ast )=0$; its posterior is ${\delta }_0\otimes {\delta }_0$.

Under the full posterior in step 1.1, $\mathrm{Pr}({\theta }_1=1)=\mathrm{Pr}({\theta }_2=1)=1\mathrm{/}2$ but $\mathrm{Pr}({\theta }_1=1,{\theta }_2=1)=1\mathrm{/}2\mathrm{\ne }1\mathrm{/}4$. Therefore the group parameters are dependent after the shared hyperparameter is integrated out, and the full posterior differs from the plug-in posterior in step 2.1.

By [A1, F3], $\lambda ([0,1])=1$, so the uniform prior is proper. Under $M_0$, $m_0(0)=1$. Under $M_1$, its density is $1$ on $[0,1]$, so $m_1(0)={\int }_0^1(1-\theta )d\theta =1\mathrm{/}2$. Both evidences are finite and positive, and [F1] gives $\mathrm{Pr}(M_0\mid y=0)=w_0\mathrm{/}(w_0+w_1\mathrm{/}2)>0$.

By [F3], the degenerate closed interval $[0,0]=\{0\}$ is Lebesgue measurable and has measure zero; [F4] identifies this measure as $\lambda$. Thus the uniform prior assigns the singleton mass ${\int }_{\{0\}}1d\lambda =0$. The separate model atom in step 1.1 instead gives $\theta =0$ prior weight $w_0$ and positive posterior probability.