Type-S density posteriors satisfy the weak multiscale Bernstein–von Mises theorem

Choose a deterministic $b_n\to \mathrm{\infty }$ sufficiently slowly that $b_n\sqrt{d}{e}_n\to 0$ and $b_n{e}_n{d}^{2r+1}\mathrm{/}\sqrt{n}\to 0$ in the Gaussian case. Such a choice exists: the first unmultiplied expression tends to zero because $\alpha >1\mathrm{/}2$, and the second is bounded by a logarithmic factor times $d^{2r-2\alpha +1\mathrm{/}2}\to 0$. Set $D_n=\{\mathrm{\parallel }\theta -{\theta }_{\ast }{\mathrm{\parallel }}_2\le b_n{e}_n\}$ in the quotient by constants used in [F1]. Then $\mathrm{\Pi }(D_n\mid X^n)\to 1$ in probability. On $D_n$, $\mathrm{\parallel }\theta -{\theta }_{\ast }{\mathrm{\parallel }}_{\mathrm{\infty }}\le C{b}_n\sqrt{d}{e}_n=o(1)$; hence $\mathrm{\parallel }f-f_0{\mathrm{\parallel }}_{\mathrm{\infty }}=o(1)$ uniformly, and the densities are bounded above and away from zero. Write ${\mathrm{\Pi }}_D$ for the posterior conditioned on $D_n$.

For $\gamma ={\psi }_{lk}$, $l\le L_n$, define $f_s=f{e}^{-s\gamma \mathrm{/}\sqrt{n}}\mathrm{/}{P}_f{e}^{-s\gamma \mathrm{/}\sqrt{n}}$. For $\mathrm{\mid }s\mathrm{\mid }\le \sqrt{{\lambda }_l}$, Taylor expansion of the log moment generating function gives $s\sqrt{n}(P_f-P_n)\gamma ={\mathrm{\ell }}_n(f_s)-{\mathrm{\ell }}_n(f)+(s^2\mathrm{/}2){\mathrm{Var}}_f\gamma +R$, where $\mathrm{\mid }R\mathrm{\mid }\le C\mathrm{\mid }s{\mathrm{\mid }}^3\mathrm{\parallel }\gamma {\mathrm{\parallel }}_{\mathrm{\infty }}\mathrm{/}\sqrt{n}$. To justify the uniform remainder, differentiate $a\mapsto \log P_f{e}^{a\gamma }$ three times; its third derivative is the third centered moment under the tilted density, bounded by $C\mathrm{\parallel }\gamma {\mathrm{\parallel }}_{\mathrm{\infty }}{P}_f{\gamma }^2$ when $\mathrm{\mid }a\mathrm{\mid }\mathrm{\parallel }\gamma {\mathrm{\parallel }}_{\mathrm{\infty }}=o(1)$. On $D_n$, $P_f{\gamma }^2\le C$ by orthonormality and the density bound, while $\sqrt{L_n}{2}^{L_n\mathrm{/}2}\mathrm{/}\sqrt{n}\to 0$. Thus $\mathrm{\mid }R\mathrm{\mid }\le C{s}^2$ and ${\mathrm{Var}}_f\gamma \le C$. Here ${\mathrm{\ell }}_n$ is the sum log likelihood.

The map $f\mapsto f_s$ is the quotient-coordinate translation $\theta \mapsto \theta -s(\gamma -\int \gamma )\mathrm{/}\sqrt{n}$. On high wavelets it shifts one coordinate; low scaling directions involve only a fixed finite-dimensional block. In S1 the absolute log prior density ratio is at most $C\mathrm{\mid }s\mathrm{\mid }\mathrm{/}(\sqrt{n}{\sigma }_l)\le C\mathrm{\mid }s\mathrm{\mid }$, since $2^{L_n(\alpha +1\mathrm{/}2)}\le \sqrt{n}$. In S2 it is at most $C[\mathrm{\mid }s\mathrm{\mid }(\mathrm{\mid }{g}_{0,lk}\mathrm{\mid }+b_n{e}_n)\mathrm{/}(\sqrt{n}{\sigma }_l^2)+s^2\mathrm{/}(n{\sigma }_l^2)]$, with a bounded fixed low-block version. Uniformly in high $l\le L_n$, its coefficients are bounded: $\mathrm{\mid }{g}_{0,lk}\mathrm{\mid }\le C{2}^{-l(\alpha +1\mathrm{/}2)}$, $b_n{e}_n{d}^{2r+1}\mathrm{/}\sqrt{n}=o(1)$, and $d^{2r+1}\mathrm{/}n=o(1)$. Changing variables in the numerator likelihood integral and discarding its transformed $D_n$ indicator therefore bounds that integral by $e^{C\mathrm{\mid }s\mathrm{\mid }+C{s}^2}$ times the full posterior normalizer. Step 2.1 yields $E_{{\mathrm{\Pi }}_D}{e}^{s\sqrt{n}(P_f-P_n){\psi }_{lk}}\le C{e}^{C{s}^2+C\mathrm{\mid }s\mathrm{\mid }}\mathrm{/}\mathrm{\Pi }(D_n\mid X^n)$. No translation invariance of $D_n$ is used.

Put $Z_{lk}=\sqrt{n}⟨f-P_n,{\psi }_{lk}⟩$ for $l\le L_n$ and $p_n=\mathrm{\Pi }(D_n\mid X^n)$. Exponential Markov with $s=\pm \sqrt{{\lambda }_l}$ gives ${\mathrm{\Pi }}_D\{\mathrm{\mid }{Z}_{lk}\mathrm{\mid }>u\sqrt{{\lambda }_l}\}\le C{p}_n^{-1}{e}^{-{\lambda }_l(u-C)}$. Union over the at most $C{2}^l$ locations shows $E_{{\mathrm{\Pi }}_D}{\max }_k\mathrm{\mid }{Z}_{lk}\mathrm{\mid }\le C{p}_n^{-1}\sqrt{{\lambda }_l}$. Union also over levels and integrate in $u$ above a sufficiently large constant to obtain $E_{{\mathrm{\Pi }}_D}{\sup }_{l\le L_n,k}\mathrm{\mid }{Z}_{lk}\mathrm{\mid }\mathrm{/}\sqrt{{\lambda }_l}\le C\mathrm{/}{p}_n$: the relevant integral is bounded by $C{p}_n^{-1}{\sum }_{l\ge 1}{2}^l{\int }_M^{\mathrm{\infty }}{e}^{-l(u-C)}du$, plus a finite low-level term. Also the wavelet level bound and a geometric sum give $E_{{\mathrm{\Pi }}_D}\mathrm{\parallel }{P}_{L_n}(f-P_n){\mathrm{\parallel }}_{\mathrm{\infty }}\le C{p}_n^{-1}\sqrt{d{\lambda }_{L_n}\mathrm{/}n}$.

Fix a finite linear combination $\gamma$ of wavelets and a fixed real $s$. Repeat step 2.1 with this bounded $\gamma$. Its remainder is now $o(1)$, and ${\mathrm{Var}}_f\gamma \to {\mathrm{Var}}_0\gamma$ uniformly on $D_n$. The prior translation involves finitely many fixed coordinates, and its density ratio tends to one uniformly on $D_n$: use the global S1 log-Lipschitz bound or the fixed Gaussian precision block and bounded quotient coordinates in S2. Let $A_s$ be the translated image of $D_n$. For large $n$, $A_s$ contains $D_n^{-}:=\{\mathrm{\parallel }\theta -{\theta }_{\ast }{\mathrm{\parallel }}_2\le b_n{e}_n\mathrm{/}2\}$ because the translation has norm $O(n^{-1\mathrm{/}2})=o(b_n{e}_n)$. Both $\mathrm{\Pi }(D_n^{-}\mid X^n)$ and $\mathrm{\Pi }(D_n\mid X^n)$ tend to one by [F1]. Bounding the translated integral above by the full normalizer and below by its integral on $D_n^{-}$ therefore gives $E_{{\mathrm{\Pi }}_D}{e}^{s\sqrt{n}(P_f-P_n)\gamma }\to e^{s^2{\mathrm{Var}}_0(\gamma )\mathrm{/}2}$ in probability.

Each empirical wavelet coordinate has variance at most $D_0\mathrm{/}n$ and summands bounded by $C{2}^{l\mathrm{/}2}$. Bernstein [F5] gives $\mathrm{Pr}\{\mathrm{\mid }(P_n-P_0){\psi }_{lk}\mathrm{\mid }>C[\sqrt{t\mathrm{/}n}+2^{l\mathrm{/}2}t\mathrm{/}n]\}\le 2e^{-t}$. Choose $t=C^{\mathrm{\prime }}(l+1)+u$, union over $k$, and integrate over $u\ge 0$ to get $E{\max }_k\mathrm{\mid }(P_n-P_0){\psi }_{lk}\mathrm{\mid }\le C[\sqrt{{\lambda }_l\mathrm{/}n}+2^{l\mathrm{/}2}{\lambda }_l\mathrm{/}n]$. Summing against $2^{l\mathrm{/}2}$ yields $\mathrm{\parallel }{P}_{L_n}(P_n-P_0){\mathrm{\parallel }}_{\mathrm{\infty }}=O_P(\sqrt{d{\lambda }_{L_n}\mathrm{/}n}+d{\lambda }_{L_n}\mathrm{/}n)=O_P(\sqrt{d{\lambda }_{L_n}\mathrm{/}n})$, since $d{\lambda }_{L_n}\mathrm{/}n\to 0$. Moreover $\mathrm{\parallel }{P}_{L_n}{f}_0-f_0{\mathrm{\parallel }}_{\mathrm{\infty }}\le C{d}^{-\alpha }\le C\sqrt{d{\lambda }_{L_n}\mathrm{/}n}$ for both cutoffs. Consequently the posterior projected density error is tight at rate $a_n=\sqrt{d{\lambda }_{L_n}\mathrm{/}n}$.

Here is the needed finite-dimensional Laplace continuity argument. Along any subsequence, extract a further subsequence on which the preceding transform convergences hold almost surely for every rational $s$ and every rational vector of finite-coordinate coefficients, using countability and convergence in probability. The transforms at the positive and negative coordinate unit vectors give tightness of these deterministic finite-dimensional laws by exponential Markov and a finite union bound. [F6] gives a weakly convergent further subsequence. For a fixed rational direction and rational $s$, its exponential is uniformly integrable: its second moment is bounded by the transform at $2s$, and Cauchy–Schwarz and Markov bound the expectation above any large cutoff. Truncation thus passes its transform to the weak limit. In each coordinate direction and then any rational linear direction, the limit has finite exponential moments of every rational order, and hence of every real order by bounding $e^{t\mathrm{\mid }x\mathrm{\mid }}$ by $e^{qx}+e^{-qx}$ for rational $q>\mathrm{\mid }t\mathrm{\mid }$. Its moment generating function is analytic: on every bounded complex disk its power series can be integrated term by term under $e^{q\mathrm{\mid }x\mathrm{\mid }}$. Equality with the Gaussian transform on the rationals therefore gives equality of all moments by the real power series at zero. Integrating the characteristic-function power series under the same exponential bound gives the Gaussian characteristic function. [F6] identifies every rational linear projection with the corresponding centered normal. Continuity of characteristic functions in their vector argument extends the identification to real vectors; [F6] then identifies the finite-dimensional law with the Gaussian projection of [F3]. Every weakly convergent further subsequence has that limit, so tightness implies weak convergence of the full deterministic subsequence, and [F7] gives bounded-Lipschitz convergence. The initial subsequence argument proves convergence in probability of the original random projected laws.

For the coefficients through $L_n$, step 4.1 and Markov give posterior tightness of ${\sup }_{l\le L_n,k}\mathrm{\mid }{Z}_{lk}\mathrm{\mid }\mathrm{/}\sqrt{{\lambda }_l}$. Therefore for $J<L_n$, the $M(w)$ norm of the part with $J<l\le L_n$ is at most this variable times ${\sup }_{l>J}\sqrt{{\lambda }_l}\mathrm{/}{w}_l$, which tends to zero as $J\to \mathrm{\infty }$ by admissibility. This proves the required uniform projected posterior tail bound.

This projected rate controls the full supremum error. Put $h=\theta -{\theta }_{\ast }$, $f_{\ast }=e^{{\theta }_{\ast }}\mathrm{/}\int e^{{\theta }_{\ast }}$, and $v=h-P_{f_{\ast }}h\in V_{L_n}$. Since $\int h=0$, $\mathrm{\parallel }h{\mathrm{\parallel }}_{\mathrm{\infty }}\le 2\mathrm{\parallel }v{\mathrm{\parallel }}_{\mathrm{\infty }}$. Normalized exponential expansion gives $f-f_{\ast }=f_{\ast }v+R$ with $\mathrm{\parallel }R{\mathrm{\parallel }}_{\mathrm{\infty }}\le C\mathrm{\parallel }h{\mathrm{\parallel }}_{\mathrm{\infty }}^2$ on $D_n$. Also $\mathrm{\parallel }{f}_{\ast }-f_0{\mathrm{\parallel }}_{\mathrm{\infty }}\le C{d}^{-\alpha }$. For any fixed $0<\beta <\min (\alpha ,1)$, $f_0$ is $\beta$-Hölder, so [F2] gives $\mathrm{\parallel }{P}_{L_n}(f_{0}v)-f_{0}v{\mathrm{\parallel }}_{\mathrm{\infty }}\le C{d}^{-\beta }\mathrm{\parallel }v{\mathrm{\parallel }}_{\mathrm{\infty }}$. The lower bound $f_0\ge {\rho }_0$ and uniform boundedness of $P_{L_n}$ imply $c\mathrm{\parallel }h{\mathrm{\parallel }}_{\mathrm{\infty }}\le \mathrm{\parallel }{P}_{L_n}(f-f_0){\mathrm{\parallel }}_{\mathrm{\infty }}+C{d}^{-\beta }\mathrm{\parallel }h{\mathrm{\parallel }}_{\mathrm{\infty }}+C\mathrm{\parallel }h{\mathrm{\parallel }}_{\mathrm{\infty }}^2+C{d}^{-\alpha }$. Step 1.1 makes the two terms proportional to $\mathrm{\parallel }h{\mathrm{\parallel }}_{\mathrm{\infty }}$ absorbable. It follows that $\mathrm{\parallel }f-f_0{\mathrm{\parallel }}_{\mathrm{\infty }}\le C[\mathrm{\parallel }{P}_{L_n}(f-f_0){\mathrm{\parallel }}_{\mathrm{\infty }}+d^{-\alpha }]$ uniformly on $D_n$. Thus the full posterior supremum error is tight at rate $a_n$.

Choose $M_n\to \mathrm{\infty }$ so slowly that $M_n\sqrt{{\lambda }_{L_n}}\mathrm{/}{w}_{L_n}\to 0$. Step 6.1 makes the posterior probability of $\mathrm{\parallel }f-f_0{\mathrm{\parallel }}_{\mathrm{\infty }}>M_n{a}_n$ tend to zero. On its complement, the $L^1$ wavelet bound and monotonicity of $w$ give $\sqrt{n}\mathrm{\parallel }{P}_{>L_n}(f-f_0){\mathrm{\parallel }}_{M(w)}\le C\sqrt{n}{2}^{-L_n\mathrm{/}2}{w}_{L_n}^{-1}{M}_n{a}_n\le C{M}_n\sqrt{{\lambda }_{L_n}}\mathrm{/}{w}_{L_n}\to 0$. The truth coefficients satisfy $\sqrt{n}\mathrm{\parallel }{P}_{>L_n}{f}_0{\mathrm{\parallel }}_{M(w)}\le C\sqrt{n}{d}^{-\alpha -1\mathrm{/}2}\mathrm{/}{w}_{L_n}\le C\sqrt{{\lambda }_{L_n}}\mathrm{/}{w}_{L_n}\to 0$ for either cutoff. Hence the entire density tail above $L_n$ is negligible in posterior probability.

Each density is a finite exponential of bounded continuous functions, hence belongs to $L^2$; its coefficients, and those of $P_n(L_n)$, belong to $M_0(w)$. Removing $D_n$ changes posterior integrals of functions bounded by one by at most $2[1-\mathrm{\Pi }(D_n\mid X^n)]=o_P(1)$. Step 5.2 supplies fixed finite-projection convergence to [F3], and steps 5.3, 7.1 supply the posterior-tail condition. Apply [F4] to conclude the asserted bounded-Lipschitz convergence in $M_0(w)$.