Misspecified M-estimator sandwich limit

Put ${\mathrm{\Delta }}_n={\sup }_{\theta \in \mathrm{\Theta }}\mathrm{\mid }{P}_n{\mathrm{\ell }}_{\theta }-M(\theta )\mathrm{\mid }$ and $r_n={\sup }_{\theta \in \mathrm{\Theta }}{P}_n{\mathrm{\ell }}_{\theta }-P_n{\mathrm{\ell }}_{{\widehat{\theta }}_n}$. By assumption ${\mathrm{\Delta }}_n{\to }_{P}0$ and $r_n{\to }_{P}0$. Since ${\widehat{\theta }}_n$ is an approximate maximizer by [F8], $P_n{\mathrm{\ell }}_{{\widehat{\theta }}_n}\ge P_n{\mathrm{\ell }}_{{\theta }^{\ast }}-r_n$, and therefore $0\le M({\theta }^{\ast })-M({\widehat{\theta }}_n)\le 2{\mathrm{\Delta }}_n+r_n$.

For any fixed $\epsilon >0$, separation gives a positive gap ${\eta }_{\epsilon }$ whenever $\mathrm{\parallel }{\widehat{\theta }}_n-{\theta }^{\ast }{\mathrm{\parallel }}_2\ge \epsilon$. Step 1.1 then gives $P(\mathrm{\parallel }{\widehat{\theta }}_n-{\theta }^{\ast }{\mathrm{\parallel }}_2\ge \epsilon )\le P(2{\mathrm{\Delta }}_n+r_n\ge {\eta }_{\epsilon })\to 0$. Thus ${\widehat{\theta }}_n-{\theta }^{\ast }=o_P(1)$ by [F6], so with probability tending to one the segment from ${\theta }^{\ast }$ to ${\widehat{\theta }}_n$ lies in any fixed neighborhood on which the score is differentiable.

Choose a ball about ${\theta }^{\ast }$ inside the differentiability neighborhood; the measurable event that ${\widehat{\theta }}_n$ lies in this ball ensures its segment from ${\theta }^{\ast }$ stays in that neighborhood. On this event define $A_n={\int }_0^1{P}_n{\mathrm{\nabla }}_{\theta }{S}_{{\theta }^{\ast }+t({\widehat{\theta }}_n-{\theta }^{\ast })}dt$; set $A_n=H$ otherwise. For each fixed parameter in a smaller ball, density measurability makes the log-density measurable, and difference quotients make the sections of $S_{\theta }$ and ${\mathrm{\nabla }}_{\theta }{S}_{\theta }$ measurable. On compact subballs, rounding parameters to rational grids gives countable-valued Borel approximations; continuity in $\theta$ makes the score and Hessian fields jointly measurable as pointwise limits. The segment integrand is then measurable, and its continuous sample paths make $A_n$ the limit of measurable Riemann sums. Given $\epsilon >0$, continuity of $D$ at ${\theta }^{\ast }$ supplies a sufficiently small fixed $\delta$ for which $\mathrm{\parallel }D(\theta )-H{\mathrm{\parallel }}_{op}<\epsilon \mathrm{/}2$ throughout the $\delta$-ball. Consistency from step 2.1 and the assumed uniform Hessian law imply $P(\mathrm{\parallel }{A}_n-H{\mathrm{\parallel }}_{op}>\epsilon )\to 0$.

On the same high-probability event, the componentwise fundamental theorem of calculus along the segment gives $P_n{S}_{{\widehat{\theta }}_n}=P_n{S}_{{\theta }^{\ast }}+A_n({\widehat{\theta }}_n-{\theta }^{\ast })$. By step 3.1, every entry of $A_n-H$ tends to zero in probability. The determinant and adjugate are polynomials in the entries, so $\det A_n{\to }_P\det H\mathrm{\ne }0$ and $\mathrm{adj}(A_n){\to }_P\mathrm{adj}(H)$; on the event $\det A_n\mathrm{\ne }0$, $A_n^{-1}=\mathrm{adj}(A_n)\mathrm{/}\det A_n$, hence every entry of $A_n^{-1}-H^{-1}$ tends to zero in probability. The finite-dimensional matrix bound $\mathrm{\parallel }B{\mathrm{\parallel }}_{op}\le d{\max }_{j,k}\mathrm{\mid }{B}_{jk}\mathrm{\mid }$ then gives $\mathrm{\parallel }{A}_n^{-1}-H^{-1}{\mathrm{\parallel }}_{op}{\to }_{P}0$. Rearranging gives $\sqrt{n}({\widehat{\theta }}_n-{\theta }^{\ast })=-A_n^{-1}\sqrt{n}{P}_n{S}_{{\theta }^{\ast }}+A_n^{-1}\sqrt{n}{P}_n{S}_{{\widehat{\theta }}_n}$ on an event whose probability tends to one.

By [F2], the score vectors $S_{{\theta }^{\ast }}(X_i)$ are IID; their mean is zero and their covariance is $J$. Their second moment is finite by the Statement, so [F4] and [A1] give $\sqrt{n}{P}_n{S}_{{\theta }^{\ast }}\Rightarrow G$ with $G\sim N_d(0,J)$. The matrix $A_n^{-1}$ converges in probability in operator norm to $H^{-1}$, and the residual vector $\sqrt{n}{P}_n{S}_{{\widehat{\theta }}_n}=o_P(1)$ by the Statement. Flattening the matrix entries and concatenating the vectors gives a finite-dimensional vector converging in probability to $(H^{-1},0)$. Applying [F5] to the continuous map $(x,A,r)\mapsto -Ax+Ar$ gives the limit $-H^{-1}G$. Its covariance is $H^{-1}J{H}^{-T}$, proving the assertion.