Parametric Bernstein–von Mises limits and misspecification

Let $E$ be the common measurable $P$-full set in the hypothesis, intersected with $\{M<\mathrm{\infty }\}$. Replace every $f_t$ and $M$ by zero off $E$. This preserves all integrals and every sample path on the measurable probability-one event $\{X_i\in E\text{ for all }i\}$, and gives the Lipschitz bound everywhere. The countable-net suprema below are therefore measurable versions of the original suprema. A radius-$r$ Euclidean ball has finite $r{2}^{-k}$-nets $T_k$ with $\mathrm{\mid }{T}_k\mathrm{\mid }\le (C_d{2}^k{)}^d$ for a finite dimension-dependent constant $C_d$: cover a containing cube by a grid of side at most $r{2}^{-k}\mathrm{/}\sqrt{d}$ and use closed grid cells; from each nonempty compact intersection with the ball, take its lexicographically least point, obtained by successive coordinate minima. The union of these nets is countable and dense. For each fixed sample in the common full set, $t\mapsto f_t(X_i)$ is Lipschitz and hence continuous, so the supremum over the ball equals the supremum over that countable union and is measurable; the centered paths are continuous too because $\mathrm{\mid }P{f}_t-P{f}_s\mathrm{\mid }\le PM\mathrm{\parallel }t-s{\mathrm{\parallel }}_2$. The same grid argument on the compact pair domain makes the pairwise-modulus supremum measurable. The first supremum is integrable because $\mathrm{\mid }{f}_t\mathrm{\mid }\le rM$ and $PM<\mathrm{\infty }$.

Conditional on $X_1,\dots ,X_n$, write $R_n(t)=n^{-1\mathrm{/}2}{\sum }_i{\epsilon }_i{f}_t(X_i)$, with signs on the product probability space [F2]. Since $E{e}^{\lambda \epsilon a}=\cosh (\lambda a)\le e^{{\lambda }^2{a}^2\mathrm{/}2}$, its increments satisfy $E_{\epsilon }{e}^{\lambda (R_n(t)-R_n(s))}\le e^{{\lambda }^2(P_n{M}^2)\mathrm{\parallel }t-s{\mathrm{\parallel }}_2^2\mathrm{/}2}$. The inequality for $\cosh$ follows by integrating $(\log \cosh u{)}^{\mathrm{\prime }}=\tanh u\le u$ for $u\ge 0$ and using evenness. Thus each increment is sub-Gaussian with scale $(P_n{M}^2{)}^{1\mathrm{/}2}\mathrm{\parallel }t-s{\mathrm{\parallel }}_2$.

By [F1] and [F2], adjoin an independent IID copy $X_1^{\mathrm{\prime }},\dots ,X_n^{\mathrm{\prime }}$ and independent signs ${\epsilon }_i\in \{-1,1\}$, each equally likely. For each fixed original sample, the supremum of absolute values of affine functions of the ghost sample is convex; Jensen's inequality therefore bounds $E{\sup }_t\mathrm{\mid }\sqrt{n}(P_n-P)f_t\mathrm{\mid }$ by $E{\sup }_t\mathrm{\mid }{n}^{-1\mathrm{/}2}{\sum }_i(f_t(X_i)-f_t(X_i^{\mathrm{\prime }}))\mathrm{\mid }$. Each pair $(X_i,X_i^{\mathrm{\prime }})$ is exchangeable, so independently swapping the members of each pair with sign ${\epsilon }_i$ preserves the joint law. The triangle inequality then bounds the last expectation by $2E{E}_{\epsilon }{\sup }_t\mathrm{\mid }{n}^{-1\mathrm{/}2}{\sum }_i{\epsilon }_i{f}_t(X_i)\mathrm{\mid }$.

Set $T_0=\{0\}$ and fix an ordering of each finite net $T_k$. For each $k\ge 1$, let ${\pi }_k(t)$ be the first nearest point in that ordering; then $\mathrm{\parallel }{\pi }_k(t)-{\pi }_{k-1}(t){\mathrm{\parallel }}_2\le 3r{2}^{-k}$. For $N$ centered sub-Gaussian variables of scale at most $\sigma$, the exponential-moment bound gives $E{\max }_{j\le N}\mathrm{\mid }{Y}_j\mathrm{\mid }\le \log (2N)\mathrm{/}\lambda +\lambda {\sigma }^2\mathrm{/}2$ for every $\lambda >0$; the case $\sigma =0$ is immediate, and otherwise minimizing at $\lambda =\sqrt{2\log (2N)}\mathrm{/}\sigma$ gives $\sigma \sqrt{2\log (2N)}$. Apply this to the at most $\mathrm{\mid }{T}_k\mathrm{\mid }\mathrm{\mid }{T}_{k-1}\mathrm{\mid }$ links and sum the resulting bounds: $E_{\epsilon }{\sup }_t\mathrm{\mid }{R}_n(t)\mathrm{\mid }\le C_{d}r(P_n{M}^2{)}^{1\mathrm{/}2}{\sum }_{k\ge 1}{2}^{-k}\sqrt{k}\le C_d^{\mathrm{\prime }}r(P_n{M}^2{)}^{1\mathrm{/}2}$. Continuity of the sample paths passes from the nets to the full ball.

Combine the symmetrization bound in step 2.1 with step 2.2 and use $E(P_n{M}^2{)}^{1\mathrm{/}2}\le (P{M}^2{)}^{1\mathrm{/}2}$, which is Cauchy--Schwarz. Absorbing the numerical factors into $C_d$ proves the first expectation inequality.

For the modulus bound, apply the ghost-sample and independent-swap argument of step 2.1 to the class $g_{s,t}=f_t-f_s$ indexed by the compact set $D_{r,\epsilon }=\{(s,t):\mathrm{\parallel }s{\mathrm{\parallel }}_2,\mathrm{\parallel }t{\mathrm{\parallel }}_2\le r,\mathrm{\parallel }t-s{\mathrm{\parallel }}_2\le \epsilon \}$. Conditional on the data, its signed process is $U_n(s,t)=R_n(t)-R_n(s)$; its increments between $(s,t)$ and $(s^{\mathrm{\prime }},t^{\mathrm{\prime }})$ are sub-Gaussian with scale at most $(P_n{M}^2{)}^{1\mathrm{/}2}(\mathrm{\parallel }s-s^{\mathrm{\prime }}{\mathrm{\parallel }}_2+\mathrm{\parallel }t-t^{\mathrm{\prime }}{\mathrm{\parallel }}_2)$ by step 1.2. At pair-mesh $\epsilon 2^{-k}$, grid the ambient product ball, use closed cells and their lexicographically least points in nonempty compact intersections with $D_{r,\epsilon }$, and adjoin $(0,0)$. The resulting finite nets have at most $(Cr{2}^k\mathrm{/}\epsilon {)}^{2d}$ points after increasing the dimension-dependent $C$. The maximum on the initial net has conditional expectation at most $(P_n{M}^2{)}^{1\mathrm{/}2}\epsilon \sqrt{2\log (2N_0)}$, since every pair in $D_{r,\epsilon }$ has $\mathrm{\parallel }t-s{\mathrm{\parallel }}_2\le \epsilon$ and $U_n(0,0)=0$. Between consecutive nets, the link lengths are at most a constant times $\epsilon 2^{-k}$ and the number of links is at most $N_k{N}_{k-1}$. The sub-Gaussian maximum bound from step 2.2 therefore bounds the expected maximum link at level $k$ by $C_d(P_n{M}^2{)}^{1\mathrm{/}2}\epsilon 2^{-k}\sqrt{\log (er\mathrm{/}\epsilon )+k}$. Summing over $k\ge 1$ gives $C_d(P_n{M}^2{)}^{1\mathrm{/}2}\epsilon \sqrt{\log (er\mathrm{/}\epsilon )}$; continuity extends the net bound to $D_{r,\epsilon }$. Symmetrization and $E(P_n{M}^2{)}^{1\mathrm{/}2}\le (P{M}^2{)}^{1\mathrm{/}2}$ prove the second inequality, after increasing $C_d$.

Let $Q_n$ be the posterior law of $H_n$. The contraction premise implies that $(Q_n)$ is tight in probability: for every $\eta ,\epsilon >0$ there is a finite $R$ such that ${\mathrm{lim\hspace{0.17em}sup}}_n{P}_{0,n}\{Q_n(\mathrm{\parallel }h{\mathrm{\parallel }}_2>R)>\eta \}<\epsilon$. Otherwise there would be $\eta ,\epsilon >0$ and increasing radii $R_k\to \mathrm{\infty }$ with indices $n_k$ for which $P_{0,n_k}\{Q_{n_k}(\mathrm{\parallel }h{\mathrm{\parallel }}_2>R_k)>\eta \}>\epsilon$. Define $M_n=R_k$ for $n_k\le n<n_{k+1}$, with any positive initial values. Then $M_n\to \mathrm{\infty }$ and the assumed contraction fails along $n_k$, a contradiction.

The supplied common-dominator density setup [F1, F2] makes the posterior likelihood ratio well-defined on the stated $P_{0,n}$-full set. Fix $R>0$ and put $K_R=\{h:\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R\}$. Since ${\theta }^{\ast }$ is interior and ${\delta }_n\to 0$, ${\theta }^{\ast }+{\delta }_n{K}_R\subset \mathrm{\Theta }$ for all sufficiently large $n$. Relative to Lebesgue measure on $K_R$, the unnormalized posterior density for $H_n$ is proportional to $q_n(h)=\exp ({\mathrm{\ell }}_n({\theta }^{\ast }+{\delta }_{n}h))\pi ({\theta }^{\ast }+{\delta }_{n}h)$; the Jacobian ${\delta }_n^d$ cancels on conditioning on $K_R$. Set $g_n(h)=\exp (h^T{\mathrm{\Delta }}_n-\frac{1}{2}{h}^{T}Vh)$. By compact-uniform LAN and continuity of $\pi$ at ${\theta }^{\ast }$, $q_n(h)\mathrm{/}{g}_n(h)=\pi ({\theta }^{\ast })(1+o_{P_{0,n}}(1))$ uniformly on $K_R$. Completing the square gives $g_n$ proportional to the density in [F5] with mean $V^{-1}{\mathrm{\Delta }}_n$ and covariance $V^{-1}$. If $w_n=q_n\mathrm{/}{g}_n$ and ${\bar{w}}_n={\int }_{K_R}{w}_{n}d{G}_n^{K_R}$, where $G_n^{K_R}$ is the Gaussian law conditioned on $K_R$, the two conditional densities have ratio $w_n\mathrm{/}{\bar{w}}_n$. Uniform convergence $w_n\mathrm{/}\pi ({\theta }^{\ast })\to 1$ implies ${\bar{w}}_n\mathrm{/}\pi ({\theta }^{\ast })\to 1$, so ${\sup }_{K_R}\mathrm{\mid }{w}_n\mathrm{/}{\bar{w}}_n-1\mathrm{\mid }=o_{P_{0,n}}(1)$ and their total-variation distance tends to zero.

By [F6] and the assumed ${\mathrm{\Delta }}_n=O_{P_{0,n}}(1)$, the Gaussian means $V^{-1}{\mathrm{\Delta }}_n$ are bounded in probability. For any fixed bound on the mean, its Gaussian probability outside $K_R$ is at most the centered $N_d(0,V^{-1})$ probability outside a ball whose radius tends to infinity with $R$. For integers $m\ge 1$, the sets $F_m=\{z:\mathrm{\parallel }z{\mathrm{\parallel }}_2>m\}$ decrease to the empty set, and $F_m={⨆}_{k\ge m}(F_k\setminus F_{k+1})$. Countable additivity makes $P(F_m)$ the tail of a convergent nonnegative series, so it tends to zero. Hence the Gaussian tails are tight in probability as $R\to \mathrm{\infty }$.

For any two probability measures $P,Q$ and measurable $K$, their restrictions conditioned on $K$ satisfy $\mathrm{\parallel }P-Q{\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}\le \mathrm{\parallel }P(\cdot \mid K)-Q(\cdot \mid K){\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}+2P(K^c)+2Q(K^c)$ whenever both conditioning probabilities are positive. Indeed, for any event $A$, split it into $A\cap K$ and $A\cap K^c$, compare the two conditional masses on $K$, and use $\mathrm{\mid }P(K)-Q(K)\mathrm{\mid }\le P(K^c)+Q(K^c)$. Choose $R$ so the posterior tail from step 1.1 and Gaussian tail from step 1.3 are arbitrarily small in probability; the conditional distance tends to zero for this fixed $R$ by step 1.2. Letting the tail tolerance decrease to zero proves the asserted full total-variation convergence under [F3].

By [A1, F1, F2], $\alpha (u)=o(\mathrm{\parallel }u{\mathrm{\parallel }}_2^2)$ and $\beta (u)=o(\mathrm{\parallel }u{\mathrm{\parallel }}_2^2)$. For fixed compact $H$ with $\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R$, this little-$o$ relation is uniform over $\mathrm{\parallel }u{\mathrm{\parallel }}_2\le R\mathrm{/}\sqrt{n}$: $n{\sup }_{h\in H}\alpha (h\mathrm{/}\sqrt{n})\to 0$ and $n{\sup }_{h\in H}\beta (h\mathrm{/}\sqrt{n})\to 0$.

Under $P_0$, each observation lies in $S$ almost surely. On $S$, the actual density ratio is $p_{{\theta }^{\ast }+u}\mathrm{/}{p}_0=(1-\alpha (u))q_u$. For $\alpha \le 1\mathrm{/}2$, $\mathrm{\mid }\log (1-\alpha )\mathrm{\mid }\le 2\alpha$, so step 1.1 gives ${\sup }_{h\in H}\mid n\log (1-\alpha (h\mathrm{/}\sqrt{n}))\mid \le 2n{\sup }_{h\in H}\alpha (h\mathrm{/}\sqrt{n})⟶0.$ Thus the actual posterior likelihood ratios have the same compact-uniform LAN expansion as the normalized proxy. The zero-overlap bound in step 1.1 is recorded separately; compact-uniform positivity is supplied by the proxy-LAN hypothesis, not inferred from pointwise QMD.

For each local $u$, write $P_{{\theta }^{\ast }+u}=(1-\alpha (u)){\bar{P}}_u+\alpha (u)S_u$, where ${\bar{P}}_u=q_u{P}_0$ is supported on $S$ and $S_u$ is the normalized restriction to $S^c$ when $\alpha (u)>0$. In the product mixture, the all-${\bar{P}}_u$ component has weight $(1-\alpha (u){)}^n$ and every other component is supported outside $S^n$. Hence $\mathrm{\parallel }{P}_{{\theta }^{\ast }+u}^{\otimes n}-{\bar{P}}_u^{\otimes n}{\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}=1-(1-\alpha (u){)}^n\le n\alpha (u)\mathrm{.}$ Step 1.1 makes this $o(1)$ uniformly for $h\in H$. This comparison is between probability laws and does not identify a full Radon–Nikodym derivative for $P_{{\theta }^{\ast }+u}^{\otimes n}$.

Step 2.1 supplies the compact-uniform LAN premise for the actual posterior likelihood ratios; the prior, evidence, and posterior-rate hypotheses are those of [F5]. For each $n$, the iid product experiment is dominated by the finite product of the common sigma-finite dominator, with product densities formed from the supplied jointly measurable versions; iterated integration on rectangles and uniqueness of product measures identify these as the densities of the iid laws. Thus the actual ratios in step 2.1 are the likelihood ratios required by [F5]. Applying [F5] with ${\delta }_n=n^{-1\mathrm{/}2}$ proves the BvM limit. The fixed-sequence assertion [F4] alone would not justify the compact supremum used here.

Define the log-ratio class to be zero off the common measurable full set of the log-Lipschitz bound, as in [F3]; this changes no integral or almost-sure sample path and makes its suprema measurable. On that full set put $f_t=\log (p_{{\theta }^{\ast }+t}\mathrm{/}{p}_{{\theta }^{\ast }})$ and $r_t=f_t-t^{T}s$. Along every coordinate direction $e_j$, $f_{a{e}_j}\mathrm{/}a\to s_j$ in $P_0$-probability as $a\to 0$, while the envelope gives $\mathrm{\mid }{f}_{a{e}_j}\mathrm{/}a\mathrm{\mid }\le M$. Dominated convergence along almost-surely convergent subsequences therefore gives $E_0{s}_j={\lim }_{a\to 0}{E}_0{f}_{a{e}_j}\mathrm{/}a=0$, since the contrast expansion makes the last numerator $O(a^2)$. The same subsequence argument applied to the directional difference quotients and the common envelope gives $\mathrm{\mid }{s}_j\mathrm{\mid }\le M$ almost surely for every coordinate, hence $\mathrm{\parallel }s{\mathrm{\parallel }}_2\le \sqrt{d}M$ and $E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2<\mathrm{\infty }$.

The contrast expansion is uniform after the local substitution: for fixed $R$, ${\sup }_{\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R}\mid n{E}_0{f}_{h\mathrm{/}\sqrt{n}}+\frac{1}{2}{h}^{T}Vh\mid ⟶0,$ because the remainder divided by $\mathrm{\parallel }t{\mathrm{\parallel }}_2^2$ tends to zero uniformly over $0<\mathrm{\parallel }t{\mathrm{\parallel }}_2\le R\mathrm{/}\sqrt{n}$.

Differentiability in probability says $r_t\mathrm{/}\mathrm{\parallel }t{\mathrm{\parallel }}_2\to 0$ in $P_0$-probability. The bound $\mathrm{\mid }{r}_t\mathrm{\mid }\mathrm{/}\mathrm{\parallel }t{\mathrm{\parallel }}_2\le M+\mathrm{\parallel }s{\mathrm{\parallel }}_2\le (1+\sqrt{d})M$ and $E_0{M}^2<\mathrm{\infty }$ imply $E_0{r}_t^2=o(\mathrm{\parallel }t{\mathrm{\parallel }}_2^2)$ by dominated convergence along every sequence $t\to 0$. Also $\mathrm{\mid }{r}_t-r_u\mathrm{\mid }\le L\mathrm{\parallel }t-u{\mathrm{\parallel }}_2$ on the common full set, where $L=(1+\sqrt{d})M$. Fix $R>0$ and $0<\epsilon \le R$, and choose a finite $\epsilon$-net $H_{\epsilon }$ of the radius-$R$ ball. For each fixed $h\in H_{\epsilon }$, independence gives $E_0{\mid n(P_n-P_0)r_{h\mathrm{/}\sqrt{n}}\mid }^2=n{\mathrm{Var}}_0(r_{h\mathrm{/}\sqrt{n}})\le n{E}_0{r}_{h\mathrm{/}\sqrt{n}}^2⟶0.$ Because the net is finite, the maximum of these centered sums tends to zero in probability. For any $h$ in the ball choose $h_{\epsilon }\in H_{\epsilon }$ with $\mathrm{\parallel }h-h_{\epsilon }{\mathrm{\parallel }}_2\le \epsilon$. The modulus bound [F3], applied to $r_t$ with envelope $L$ on radius $R\mathrm{/}\sqrt{n}$ and distance $\epsilon \mathrm{/}\sqrt{n}$, gives $E_0{\sup }_{\begin{array}{c}\mathrm{\parallel }h{\mathrm{\parallel }}_2,\mathrm{\parallel }{h}^{\mathrm{\prime }}{\mathrm{\parallel }}_2\le R\\ \mathrm{\parallel }h-h^{\mathrm{\prime }}{\mathrm{\parallel }}_2\le \epsilon \end{array}}\mid n(P_n-P_0)(r_{h\mathrm{/}\sqrt{n}}-r_{h^{\mathrm{\prime }}\mathrm{/}\sqrt{n}})\mid \le C_d\mathrm{\parallel }L{\mathrm{\parallel }}_{L^2(P_0)}\epsilon \sqrt{\log (eR\mathrm{/}\epsilon )}\mathrm{.}$ For fixed $\epsilon$, the finite-net maximum vanishes as $n\to \mathrm{\infty }$; Markov's inequality bounds the remaining modulus probability by a constant times $\epsilon \sqrt{\log (eR\mathrm{/}\epsilon )}$, which tends to zero as $\epsilon \downarrow 0$. Therefore ${\sup }_{\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R}\mid n(P_n-P_0)r_{h\mathrm{/}\sqrt{n}}\mid =o_{P_0}(1)\mathrm{.}$

For $\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R$, decompose ${\sum }_{i=1}^n{f}_{h\mathrm{/}\sqrt{n}}(X_i)=n{E}_0{f}_{h\mathrm{/}\sqrt{n}}+h^T{S}_n+n(P_n-P_0)r_{h\mathrm{/}\sqrt{n}},$ where $E_{0}s=0$ from step 1.1. Step 2.1 controls the last term uniformly, and step 1.2 gives the uniform quadratic term. Therefore the displayed LAN remainder is $o_{P_0}(1)$ uniformly on every compact $H$.

Independence, $E_{0}s=0$, and $E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2<\mathrm{\infty }$ give $E_0\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2^2=E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2$ for every $n$. Since $R^2{\mathbf{1}}_{\{\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2>R\}}\le \mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2^2$, $P_0(\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2>R)\le E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2\mathrm{/}{R}^2$, so $(S_n)$ is bounded in probability. Multiplication by the fixed matrix $V^{-1}$ preserves boundedness because $\mathrm{\parallel }{V}^{-1}{S}_n{\mathrm{\parallel }}_2\le \mathrm{\parallel }{V}^{-1}\mathrm{\parallel }\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2$. Thus ${\mathrm{\Delta }}_n=O_{P_0}(1)$ by [F2], completing the LAN assertion.

By [F1], take ${\delta }_n=n^{-1\mathrm{/}2}$ and use $S_n$ as the central vector ${\mathrm{\Delta }}_n$ in [F2]. The local LAN theorem supplies the required compact-uniform expansion with curvature $V$ and proves $S_n=O_{P_0}(1)$; [F2] therefore centers the Gaussian at $V^{-1}{S}_n$.

For each $n$, the iid sample experiment is dominated by the finite product ${\mu }^{\otimes n}$, with densities $p_{0,n}(x_1,\dots ,x_n)={\prod }_i{p}_0(x_i)$ and $p_{n,\theta }(x_1,\dots ,x_n)={\prod }_i{p}_{\theta }(x_i)$. Iterated integration gives the product-law formula on measurable rectangles, so uniqueness of product measures identifies these as densities of $P_0^{\otimes n}$ and $P_{\theta }^{\otimes n}$. Thus the LAN expansion in [F1] is the log-likelihood ratio for this iid experiment. The prior, evidence, interior-point, and parameter-tail hypotheses in the Statement are exactly the remaining hypotheses of [F2]. Applying [F2] gives the claimed total-variation limit.

Write $f_t=\log (p_{{\theta }^{\ast }+t}\mathrm{/}{p}_{{\theta }^{\ast }})$. Positive definiteness of $V$ and the contrast expansion in [F1] give constants $c,\rho >0$ with $E_0{f}_t\le -c\mathrm{\parallel }t{\mathrm{\parallel }}_2^2$ for $\mathrm{\parallel }t{\mathrm{\parallel }}_2\le \rho$. Use the measurable versions from [F3], obtained by zero extension off the common log-Lipschitz full set. The log-Lipschitz envelope and [F3] give $E_0{\sup }_{\mathrm{\parallel }t\mathrm{\parallel }\le r}\mathrm{\mid }{G}_n{f}_t\mathrm{\mid }\le Cr$ for $r\le \rho$, where $G_n=\sqrt{n}(P_n-P_0)$ and $C=C_d(E_0{M}^2{)}^{1\mathrm{/}2}$.