A local KL expansion and square-integrable log-Lipschitz envelope give misspecified iid LAN

Define the log-ratio class to be zero off the common measurable full set of the log-Lipschitz bound, as in [F3]; this changes no integral or almost-sure sample path and makes its suprema measurable. On that full set put $f_t=\log (p_{{\theta }^{\ast }+t}\mathrm{/}{p}_{{\theta }^{\ast }})$ and $r_t=f_t-t^{T}s$. Along every coordinate direction $e_j$, $f_{a{e}_j}\mathrm{/}a\to s_j$ in $P_0$-probability as $a\to 0$, while the envelope gives $\mathrm{\mid }{f}_{a{e}_j}\mathrm{/}a\mathrm{\mid }\le M$. Dominated convergence along almost-surely convergent subsequences therefore gives $E_0{s}_j={\lim }_{a\to 0}{E}_0{f}_{a{e}_j}\mathrm{/}a=0$, since the contrast expansion makes the last numerator $O(a^2)$. The same subsequence argument applied to the directional difference quotients and the common envelope gives $\mathrm{\mid }{s}_j\mathrm{\mid }\le M$ almost surely for every coordinate, hence $\mathrm{\parallel }s{\mathrm{\parallel }}_2\le \sqrt{d}M$ and $E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2<\mathrm{\infty }$.

The contrast expansion is uniform after the local substitution: for fixed $R$, ${\sup }_{\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R}\mid n{E}_0{f}_{h\mathrm{/}\sqrt{n}}+\frac{1}{2}{h}^{T}Vh\mid ⟶0,$ because the remainder divided by $\mathrm{\parallel }t{\mathrm{\parallel }}_2^2$ tends to zero uniformly over $0<\mathrm{\parallel }t{\mathrm{\parallel }}_2\le R\mathrm{/}\sqrt{n}$.

Differentiability in probability says $r_t\mathrm{/}\mathrm{\parallel }t{\mathrm{\parallel }}_2\to 0$ in $P_0$-probability. The bound $\mathrm{\mid }{r}_t\mathrm{\mid }\mathrm{/}\mathrm{\parallel }t{\mathrm{\parallel }}_2\le M+\mathrm{\parallel }s{\mathrm{\parallel }}_2\le (1+\sqrt{d})M$ and $E_0{M}^2<\mathrm{\infty }$ imply $E_0{r}_t^2=o(\mathrm{\parallel }t{\mathrm{\parallel }}_2^2)$ by dominated convergence along every sequence $t\to 0$. Also $\mathrm{\mid }{r}_t-r_u\mathrm{\mid }\le L\mathrm{\parallel }t-u{\mathrm{\parallel }}_2$ on the common full set, where $L=(1+\sqrt{d})M$. Fix $R>0$ and $0<\epsilon \le R$, and choose a finite $\epsilon$-net $H_{\epsilon }$ of the radius-$R$ ball. For each fixed $h\in H_{\epsilon }$, independence gives $E_0{\mid n(P_n-P_0)r_{h\mathrm{/}\sqrt{n}}\mid }^2=n{\mathrm{Var}}_0(r_{h\mathrm{/}\sqrt{n}})\le n{E}_0{r}_{h\mathrm{/}\sqrt{n}}^2⟶0.$ Because the net is finite, the maximum of these centered sums tends to zero in probability. For any $h$ in the ball choose $h_{\epsilon }\in H_{\epsilon }$ with $\mathrm{\parallel }h-h_{\epsilon }{\mathrm{\parallel }}_2\le \epsilon$. The modulus bound [F3], applied to $r_t$ with envelope $L$ on radius $R\mathrm{/}\sqrt{n}$ and distance $\epsilon \mathrm{/}\sqrt{n}$, gives $E_0{\sup }_{\begin{array}{c}\mathrm{\parallel }h{\mathrm{\parallel }}_2,\mathrm{\parallel }{h}^{\mathrm{\prime }}{\mathrm{\parallel }}_2\le R\\ \mathrm{\parallel }h-h^{\mathrm{\prime }}{\mathrm{\parallel }}_2\le \epsilon \end{array}}\mid n(P_n-P_0)(r_{h\mathrm{/}\sqrt{n}}-r_{h^{\mathrm{\prime }}\mathrm{/}\sqrt{n}})\mid \le C_d\mathrm{\parallel }L{\mathrm{\parallel }}_{L^2(P_0)}\epsilon \sqrt{\log (eR\mathrm{/}\epsilon )}\mathrm{.}$ For fixed $\epsilon$, the finite-net maximum vanishes as $n\to \mathrm{\infty }$; Markov's inequality bounds the remaining modulus probability by a constant times $\epsilon \sqrt{\log (eR\mathrm{/}\epsilon )}$, which tends to zero as $\epsilon \downarrow 0$. Therefore ${\sup }_{\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R}\mid n(P_n-P_0)r_{h\mathrm{/}\sqrt{n}}\mid =o_{P_0}(1)\mathrm{.}$

For $\mathrm{\parallel }h{\mathrm{\parallel }}_2\le R$, decompose ${\sum }_{i=1}^n{f}_{h\mathrm{/}\sqrt{n}}(X_i)=n{E}_0{f}_{h\mathrm{/}\sqrt{n}}+h^T{S}_n+n(P_n-P_0)r_{h\mathrm{/}\sqrt{n}},$ where $E_{0}s=0$ from step 1.1. Step 2.1 controls the last term uniformly, and step 1.2 gives the uniform quadratic term. Therefore the displayed LAN remainder is $o_{P_0}(1)$ uniformly on every compact $H$.

Independence, $E_{0}s=0$, and $E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2<\mathrm{\infty }$ give $E_0\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2^2=E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2$ for every $n$. Since $R^2{\mathbf{1}}_{\{\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2>R\}}\le \mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2^2$, $P_0(\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2>R)\le E_0\mathrm{\parallel }s{\mathrm{\parallel }}_2^2\mathrm{/}{R}^2$, so $(S_n)$ is bounded in probability. Multiplication by the fixed matrix $V^{-1}$ preserves boundedness because $\mathrm{\parallel }{V}^{-1}{S}_n{\mathrm{\parallel }}_2\le \mathrm{\parallel }{V}^{-1}\mathrm{\parallel }\mathrm{\parallel }{S}_n{\mathrm{\parallel }}_2$. Thus ${\mathrm{\Delta }}_n=O_{P_0}(1)$ by [F2], completing the LAN assertion.