Parametric Bernstein–von Mises limits and misspecification — Examples

The prior density is continuous and strictly positive at ${\theta }^{\ast }$, and ${\theta }^{\ast }$ is interior to $\mathrm{\Theta }=\mathbb{R}$. The log-likelihood ratio at ${\theta }^{\ast }+h\mathrm{/}\sqrt{n}$ is exactly $h{\mathrm{\Delta }}_n-h^2\mathrm{/}2$, where ${\mathrm{\Delta }}_n=\sqrt{n}(\bar{X}-{\theta }^{\ast })$. The standard-normal density is even, so its mean is zero; integration by parts using ${\varphi }^{\mathrm{\prime }}(x)=-x\varphi (x)$ gives $E{Z}^2=\int x^2\varphi (x)dx=\int \varphi (x)dx=1$, with boundary term $x\varphi (x)\to 0$. By [F2], the sample mean has variance $1\mathrm{/}n$, hence $E{\mathrm{\Delta }}_n^2=1$. Since $R^2{\mathbf{1}}_{\{\mathrm{\mid }{\mathrm{\Delta }}_n\mathrm{\mid }>R\}}\le {\mathrm{\Delta }}_n^2$, $P(\mathrm{\mid }{\mathrm{\Delta }}_n\mathrm{\mid }>R)\le R^{-2}$, so ${\mathrm{\Delta }}_n=O_P(1)$. The compact-uniform LAN condition holds with $V=1$.

Multiplying the likelihood by the prior gives a kernel proportional to $\exp \{-\frac{1}{2}[n(\theta -\bar{X}{)}^2+{\tau }^{-2}{\theta }^2]\}$. Completing the square yields posterior variance $v_n=(n+{\tau }^{-2}{)}^{-1}$ and posterior mean $m_n=n\bar{X}\mathrm{/}(n+{\tau }^{-2})$. Thus the scaled posterior of $H_n=\sqrt{n}(\theta -{\theta }^{\ast })$ is $N(a_n,b_n)$, where $a_n=\sqrt{n}(m_n-{\theta }^{\ast })$ and $b_n=n\mathrm{/}(n+{\tau }^{-2})$.

We have $b_n\to 1$ and $a_n-{\mathrm{\Delta }}_n=-{\tau }^{-2}\sqrt{n}\bar{X}\mathrm{/}(n+{\tau }^{-2})=o_P(1)$ because $\sqrt{n}\bar{X}=\sqrt{n}{\theta }^{\ast }+O_P(1)$. Translation invariance and the triangle inequality reduce the total-variation distance to a centered scale change and a location shift. For $b\in [1\mathrm{/}2,2]$, the centered densities ${\varphi }_b(x)=(2\pi b{)}^{-1\mathrm{/}2}{e}^{-x^2\mathrm{/}(2b)}$ are dominated by $C{e}^{-x^2\mathrm{/}4}$ and converge pointwise to ${\varphi }_1$ as $b\to 1$, so dominated convergence gives $\mathrm{\parallel }{\varphi }_b-{\varphi }_1{\mathrm{\parallel }}_1\to 0$. For a shift $u$, the fundamental theorem of calculus gives $\mathrm{\parallel }{\varphi }_b(\cdot -u)-{\varphi }_b{\mathrm{\parallel }}_1\le \mathrm{\mid }u\mathrm{\mid }\mathrm{\parallel }{\varphi }_b^{\mathrm{\prime }}{\mathrm{\parallel }}_1=C\mathrm{\mid }u\mathrm{\mid }\mathrm{/}\sqrt{b}$. These two bounds prove $\mathrm{\parallel }N(a_n,b_n)-N({\mathrm{\Delta }}_n,1){\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}=o_P(1)$ under the convention [F3].

The scaled posterior $N(a_n,b_n)$ is tight in probability because $a_n=O_P(1)$ and $b_n\le 1$. More explicitly, on $\mathrm{\mid }{a}_n\mathrm{\mid }\le R$, its mass outside $[-M_n,M_n]$ is at most $P(\mathrm{\mid }Z\mathrm{\mid }>M_n-R)$ for $Z\sim N(0,1)$; first choose $R$ by boundedness of $a_n$, then let $M_n\to \mathrm{\infty }$. Thus every-diverging-radius contraction holds, and step 2.1 directly proves the asserted posterior normal approximation.

The model and truth are dominated by Lebesgue measure. The supplied Gaussian density is normalized by [F1]. Its log likelihood is ${\mathrm{\ell }}_{\theta }(x)=-\frac{1}{2}(x-\theta {)}^2-\frac{1}{2}\log (2\pi )$, so $E_0[{\mathrm{\ell }}_0(X)-{\mathrm{\ell }}_{\theta }(X)]=\frac{1}{2}{\theta }^2$ because $E_{0}X=0$. Thus ${\theta }^{\ast }=0$ is the unique expected-log-likelihood maximizer and $V=1$. The score at zero is $s(X)=X$, hence $I=E_0{X}^2={\sigma }^2$. For $\mathrm{\mid }{\theta }_1\mathrm{\mid },\mathrm{\mid }{\theta }_2\mathrm{\mid }\le r$, $\mathrm{\mid }{\mathrm{\ell }}_{{\theta }_1}(X)-{\mathrm{\ell }}_{{\theta }_2}(X)\mathrm{\mid }\le \mathrm{\mid }{\theta }_1-{\theta }_2\mathrm{\mid }(\mathrm{\mid }X\mathrm{\mid }+r)$, a square-integrable local envelope. The local log likelihood ratio at $h\mathrm{/}\sqrt{n}$ is exactly $h\sqrt{n}\bar{X}-h^2\mathrm{/}2$. Also $E_0(\sqrt{n}\bar{X}{)}^2={\sigma }^2$, and $R^2{\mathbf{1}}_{\{\mathrm{\mid }\sqrt{n}\bar{X}\mathrm{\mid }>R\}}\le n{\bar{X}}^2$ gives $P_0(\mathrm{\mid }\sqrt{n}\bar{X}\mathrm{\mid }>R)\le {\sigma }^2\mathrm{/}{R}^2$, so $\sqrt{n}\bar{X}=O_{P_0}(1)$.

Completing the square in ${\prod }_i\varphi (X_i-\theta )\pi (\theta )$ gives posterior mean $m_n=n\bar{X}\mathrm{/}(n+{\tau }^{-2})$ and variance $v_n=(n+{\tau }^{-2}{)}^{-1}$. The empirical log likelihood is maximized at $\bar{X}$, since its $\theta$-dependent part is $n\bar{X}\theta -n{\theta }^2\mathrm{/}2$; for each $\epsilon >0$, ${\epsilon }^2{\mathbf{1}}_{\{\mathrm{\mid }\bar{X}\mathrm{\mid }>\epsilon \}}\le {\bar{X}}^2$, so $P_0(\mathrm{\mid }\bar{X}\mathrm{\mid }>\epsilon )\le {\sigma }^2\mathrm{/}(n{\epsilon }^2)$ and the estimator is consistent. The exact posterior variance satisfies $n{v}_n\to 1$, while the sample-mean variance is ${\sigma }^2\mathrm{/}n$.

The scaled posterior mean is $a_n=\sqrt{n}{m}_n=\frac{n}{n+{\tau }^{-2}}\sqrt{n}\bar{X}=O_{P_0}(1)$ and its variance parameter is $b_n=n\mathrm{/}(n+{\tau }^{-2})\le 1$. Given any deterministic $M_n\to \mathrm{\infty }$, on $\mathrm{\mid }{a}_n\mathrm{\mid }\le R$ its tail outside $[-M_n,M_n]$ is at most $P(\mathrm{\mid }Z\mathrm{\mid }>M_n-R)$ for $Z\sim N(0,1)$. First take $R$ large using boundedness in probability of $a_n$, then let $n\to \mathrm{\infty }$; this proves the every-diverging-radius posterior contraction premise.

Let $z=z_{1-\alpha \mathrm{/}2}={\mathrm{\Phi }}^{-1}(1-\alpha \mathrm{/}2)$. The exact central posterior interval is $[m_n-z\sqrt{v_n},m_n+z\sqrt{v_n}]$. Its coverage event for ${\theta }^{\ast }=0$, after multiplication by $\sqrt{n}$, is $\mathrm{\mid }\sqrt{n}\bar{X}\mathrm{\mid }\le z\sqrt{(n+{\tau }^{-2})\mathrm{/}n}$ because $m_n=n\bar{X}\mathrm{/}(n+{\tau }^{-2})$. By [F3], $\sqrt{n}\bar{X}\Rightarrow N(0,{\sigma }^2)$; the limiting interval endpoints are continuity points of this nondegenerate normal law, so [F4] gives limiting coverage $\mathrm{\Phi }(z\mathrm{/}\sigma )-\mathrm{\Phi }(-z\mathrm{/}\sigma )=2\mathrm{\Phi }(z\mathrm{/}\sigma )-1$.

Since $z>0$, the coverage in step 2.2 equals $1-\alpha$ when $\sigma =1$, is below $1-\alpha$ when $\sigma >1$, and is above $1-\alpha$ when $0<\sigma <1$. Indeed, $\mathrm{\Phi }$ is strictly increasing and $\mathrm{\Phi }(z)=1-\alpha \mathrm{/}2$. Thus the exact posterior width and sampling width disagree away from $\sigma =1$, giving nonnominal coverage.

The prior is proper because ${\int }_0^{\mathrm{\infty }}{e}^{-\theta }d\theta =1$. The local log-likelihood ratio is exactly ${\mathrm{\ell }}_n(h\mathrm{/}\sqrt{n})-{\mathrm{\ell }}_n(0)=h{\mathrm{\Delta }}_n-h^2\mathrm{/}2$ for $h\ge 0$, so the local Gaussian candidate is the untruncated $N({\mathrm{\Delta }}_n,1)$. The center ${\theta }^{\ast }=0$ is a boundary point of $\mathrm{\Theta }=[0,\mathrm{\infty })$, not an interior point.

Transform the posterior variable by $h=\sqrt{n}\theta$. Its unnormalized density on $h\ge 0$ is proportional to $\exp (h{\mathrm{\Delta }}_n-h^2\mathrm{/}2-h\mathrm{/}\sqrt{n})$. Completing the square gives $\exp [-\frac{1}{2}(h-({\mathrm{\Delta }}_n-n^{-1\mathrm{/}2}){)}^2]$. Thus the posterior is $N({\mathrm{\Delta }}_n-n^{-1\mathrm{/}2},1)$ conditioned on $[0,\mathrm{\infty })$, and assigns probability zero to $(-\mathrm{\infty },0)$.

By [F5], the total-variation distance from $N({\mathrm{\Delta }}_n,1)$ is at least its mass on $(-\mathrm{\infty },0)$, namely $\mathrm{\Phi }(-{\mathrm{\Delta }}_n)$. On $\{{\mathrm{\Delta }}_n\le 0\}$ this bound is at least $1\mathrm{/}2$. The event $(-\mathrm{\infty },0]$ is a continuity set for $N(0,1)$, so [F3] and [F4] give $P_0({\mathrm{\Delta }}_n\le 0)\to 1\mathrm{/}2$. Therefore the total-variation error is at least $1\mathrm{/}2$ with limiting probability at least $1\mathrm{/}2$, and it does not converge to zero in probability.