Likelihood Theory and Classical Calibration

Write $q=\sqrt{p_{{\theta }_0}}$ and $r_h=\sqrt{p_{{\theta }_0+h}}-q-\frac{1}{2}{h}^T{s}_{{\theta }_0}q$. By [F1], $\mathrm{\parallel }{r}_h{\mathrm{\parallel }}_2=o(\mathrm{\parallel }h\mathrm{\parallel })$. By [F3], each $q_h=\sqrt{p_{{\theta }_0+h}}$ is a probability square-root density, so $\mathrm{\parallel }{q}_h{\mathrm{\parallel }}_2^2=\mathrm{\parallel }q{\mathrm{\parallel }}_2^2=1$. Since $P_{{\theta }_0}$ is a probability and the score is square-integrable, [F2] with the constant function $1$ shows its coordinates are integrable. All products in the following norm expansion are integrable by [F2], since $q$, $r_h$, and $h^T{s}_{{\theta }_0}q$ belong to $L^2(\mu )$. Thus [F4] permits integrating the finite expansion term by term, and [F3] transfers the score term to expectation under $P_{{\theta }_0}$. Expanding $q_h=q+\frac{1}{2}{h}^T{s}_{{\theta }_0}q+r_h$ gives $0=h^T{\mathbb{E}}_{{\theta }_0}{s}_{{\theta }_0}+2⟨q,r_h⟩+\mathrm{\parallel }\frac{1}{2}{h}^T{s}_{{\theta }_0}q+r_h{\mathrm{\parallel }}_2^2$. Cauchy–Schwarz [F2] bounds the inner product by $o(\mathrm{\parallel }h\mathrm{\parallel })$; square-integrability of the score and $\mathrm{\parallel }{r}_h{\mathrm{\parallel }}_2=o(\mathrm{\parallel }h\mathrm{\parallel })$ make the last term $O(\mathrm{\parallel }h{\mathrm{\parallel }}^2)$. Thus $h^T{\mathbb{E}}_{{\theta }_0}{s}_{{\theta }_0}=o(\mathrm{\parallel }h\mathrm{\parallel })$.

For each coordinate vector $e_j$, set $h=t{e}_j$ with $t>0$ small enough that ${\theta }_0+t{e}_j\in \mathrm{\Theta }$. Divide the relation from step 1.1 by $t$ and let $t\downarrow 0$ to obtain ${\mathbb{E}}_{{\theta }_0}{s}_{{\theta }_0,j}=0$. This holds for all $j=1,\dots ,d$, hence the score vector has mean zero.

Write $P_0=P_{{\theta }_0}$, $S_0=\{p_{{\theta }_0}>0\}$, and $r_t=\sqrt{p_{{\theta }_0+t}}-\sqrt{p_{{\theta }_0}}-\frac{1}{2}{t}^T{s}_{{\theta }_0}\sqrt{p_{{\theta }_0}}$. By [F3], $\mathrm{\parallel }{r}_t{\mathrm{\parallel }}_{L^2(\mu )}=o(\mathrm{\parallel }t\mathrm{\parallel })$, and [F12] gives ${\mathbb{E}}_0{s}_{{\theta }_0}=0$. On $S_0^c$ both $\sqrt{p_{{\theta }_0}}$ and $s_{{\theta }_0}\sqrt{p_{{\theta }_0}}$ vanish, so the probability mass ${\alpha }_t={\int }_{S_0^c}{p}_{{\theta }_0+t}d\mu$ is $o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$. Expanding the QMD approximation in [F3] and applying [F9] to its cross term gives $\mathrm{\parallel }\sqrt{p_{{\theta }_0+t}}-\sqrt{p_{{\theta }_0}}{\mathrm{\parallel }}_2^2=\frac{1}{4}{t}^{T}It+o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$.

Normalize the absolutely continuous part on $S_0$ by $q_t=p_{{\theta }_0+t}{\mathbf{1}}_{S_0}\mathrm{/}(1-{\alpha }_t)$, and put $a_t=\sqrt{q_t\mathrm{/}{p}_{{\theta }_0}}-1$ on $S_0$. Since ${\alpha }_t=o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$, [F3] gives $a_t=\frac{1}{2}{t}^T{s}_{{\theta }_0}+o_{L^2(P_0)}(\mathrm{\parallel }t\mathrm{\parallel })$, so ${\mathbb{E}}_0{a}_t^2=\frac{1}{4}{t}^{T}It+o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$ and $2{\mathbb{E}}_0{a}_t+{\mathbb{E}}_0{a}_t^2=0$ from ${\mathbb{E}}_0(1+a_t{)}^2=1$. The local envelope makes $f_t=\log (p_{{\theta }_0+t}\mathrm{/}{p}_{{\theta }_0})$ square-integrable on $S_0$ and $\mathrm{\parallel }{f}_t{\mathrm{\parallel }}_2=O(\mathrm{\parallel }t\mathrm{\parallel })$. Along every sequence $t\to 0$ with $t\mathrm{/}\mathrm{\parallel }t\mathrm{\parallel }\to u$, the $L^2$ convergence of $a_t\mathrm{/}\mathrm{\parallel }t\mathrm{\parallel }$ has an almost-everywhere convergent subsequence. On it, $2\log (1+a_t)\mathrm{/}\mathrm{\parallel }t\mathrm{\parallel }\to u^T{s}_{{\theta }_0}$ pointwise. Since $\mathrm{\mid }{f}_t\mathrm{\mid }\mathrm{/}\mathrm{\parallel }t\mathrm{\parallel }\le L$ and $s_{{\theta }_0}\in L^2(P_0)$, $\mathrm{\mid }{f}_t\mathrm{/}\mathrm{\parallel }t\mathrm{\parallel }-u^T{s}_{{\theta }_0}{\mathrm{\mid }}^2\le 2L^2+2\mathrm{\parallel }{s}_{{\theta }_0}{\mathrm{\parallel }}^2$; dominated convergence [F10] gives convergence in $L^2$ along that subsequence. Every subsequence has such a subsubsequence, so convergence holds for the full sequence. Sequential compactness of the unit sphere therefore yields $f_t=t^T{s}_{{\theta }_0}+o_{L^2(P_0)}(\mathrm{\parallel }t\mathrm{\parallel })$. Moreover, the squares $a_t^2\mathrm{/}\mathrm{\parallel }t{\mathrm{\parallel }}^2$ are uniformly integrable because $a_t\mathrm{/}\mathrm{\parallel }t\mathrm{\parallel }$ converges in $L^2$ along every convergent direction. Taking the logarithm expansion in expectation requires a tail check. Fix $\epsilon >0$. On $\mathrm{\mid }{a}_t\mathrm{\mid }\le \epsilon$, the Taylor remainder $2\log (1+a_t)-2a_t+a_t^2$ is bounded by $\omega (\epsilon )a_t^2$, where $\omega (\epsilon )\to 0$. Uniform integrability gives ${\mathbb{E}}_0[a_t^2{\mathbf{1}}_{\mathrm{\mid }{a}_t\mathrm{\mid }>\epsilon }]=o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$ and therefore $P_0(\mathrm{\mid }{a}_t\mathrm{\mid }>\epsilon )=o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$. On this tail, $\mathrm{\mid }2\log (1+a_t)\mathrm{\mid }\le \mathrm{\mid }{f}_t\mathrm{\mid }+\mathrm{\mid }\log (1-{\alpha }_t)\mathrm{\mid }$; its expectation is $o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$ by the envelope, Cauchy–Schwarz [F9], and the tail probability. The tail expectations of $\mathrm{\mid }{a}_t\mathrm{\mid }$ and $a_t^2$ are also $o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$ by [F9] and uniform integrability. Letting $\epsilon \downarrow 0$ proves that the expected Taylor remainder is $o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$. Taking expectations then gives ${\mathbb{E}}_0{f}_t=2{\mathbb{E}}_0{a}_t-{\mathbb{E}}_0{a}_t^2+o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)=-\frac{1}{2}{t}^{T}It+o(\mathrm{\parallel }t{\mathrm{\parallel }}^2);$ the normalization term $\log (1-{\alpha }_t)$ is $o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$.

The squared Hellinger distance $H^2(P_{{\theta }_0+t},P_0)=\int (\sqrt{p_{{\theta }_0+t}}-\sqrt{p_{{\theta }_0}}{)}^{2}d\mu$ equals $\frac{1}{4}{t}^{T}It+o(\mathrm{\parallel }t{\mathrm{\parallel }}^2)$ by step 1.1. The affinity ${\rho }_t=\int \sqrt{p_{{\theta }_0+t}{p}_{{\theta }_0}}d\mu =1-H^2\mathrm{/}2$. On $S_0$ let $Y=\sqrt{p_{{\theta }_0+t}\mathrm{/}{p}_{{\theta }_0}}$, so ${\mathbb{E}}_{0}Y={\rho }_t$. For $y>0$, $\log y\le \log {\rho }_t+(y-{\rho }_t)\mathrm{/}{\rho }_t$ follows from $z-1-\log z\ge 0$ for $z=y\mathrm{/}{\rho }_t$, since this function has derivative $1-1\mathrm{/}z$ and its minimum is $0$ at $z=1$; integrating gives $P_0{f}_t=2{\mathbb{E}}_0\log Y\le 2\log {\rho }_t$. Since $\log (1-x)\le -x$ for $0\le x<1$ because $-x-\log (1-x)$ has derivative $x\mathrm{/}(1-x)\ge 0$ and value $0$ at $x=0$. Thus $2\log {\rho }_t\le -H^2$, so for some $c,\delta >0$, $P_0{f}_t\le -c\mathrm{\parallel }t{\mathrm{\parallel }}^2$ whenever $\mathrm{\parallel }t\mathrm{\parallel }\le {\delta }_0$.

Let ${\mathbb{G}}_{n}f=\sqrt{n}(P_{n}f-P_{0}f)$ for the empirical measure $P_n$. The pointwise Lipschitz envelope gives $\mathrm{\parallel }{f}_t-f_v{\mathrm{\parallel }}_{L^2(P_0)}\le \mathrm{\parallel }L{\mathrm{\parallel }}_2\mathrm{\parallel }t-v\mathrm{\parallel }$. The sample and mean processes are continuous in $t$ on compact local balls, so their suprema equal suprema over countable dense subsets and are measurable. To bound the local supremum, take an independent ghost sample $X_1^{\mathrm{\prime }},\dots ,X_n^{\mathrm{\prime }}$. For each fixed original sample, the supremum of absolute values is convex in the ghost sample vector, so Jensen's inequality replaces $P_{0}f$ by $n^{-1}{\sum }_{i}f(X_i^{\mathrm{\prime }})$ in the expected supremum. Swapping each pair $(X_i,X_i^{\mathrm{\prime }})$ with an independent sign and then using the triangle inequality bounds that supremum by twice the expected conditional Rademacher supremum. Condition on the observations for the latter. For the conditional Rademacher process $R_n(f)=n^{-1\mathrm{/}2}{\sum }_i{ϵ}_{i}f(X_i)$, Hoeffding's exponential bound ${\mathbb{E}}_{ϵ}{e}^{\lambda (R_n(f)-R_n(g))}={\prod }_i\cosh (\lambda (f-g)(X_i)\mathrm{/}\sqrt{n})\le \exp \{{\lambda }^2{P}_n(f-g{)}^2\mathrm{/}2\}$, since $\log \cosh u\le u^2\mathrm{/}2$ (integrate $\tanh u\le u$ for $u\ge 0$ and use evenness), makes each increment sub-Gaussian with variance proxy $P_n(f-g{)}^2\le P_n{L}^2\mathrm{\parallel }t-v{\mathrm{\parallel }}^2$. Choose successively finer dyadic Euclidean nets of the radius-$r$ parameter ball. At level $k$ the net has at most $(C{2}^k{)}^d$ points, each increment from its level-$(k-1)$ parent has scale at most $Cr{2}^{-k}(P_n{L}^2{)}^{1\mathrm{/}2}$, and the expected maximum of $N$ centered sub-Gaussian variables of scale $\sigma$ is at most $\sigma \sqrt{2\log (2N)}$, by the exponential-moment bound and optimizing $\lambda$. Summing the resulting bounds uses the convergent series ${\sum }_{k\ge 1}{2}^{-k}\sqrt{k}$ and gives ${\mathbb{E}}_{ϵ}{\sup }_{\mathrm{\parallel }t\mathrm{\parallel }\le r}\mathrm{\mid }{R}_n(f_t-f_0)\mathrm{\mid }\le C_{d}r(P_n{L}^2{)}^{1\mathrm{/}2}$. Symmetrization and $\mathbb{E}(P_n{L}^2{)}^{1\mathrm{/}2}\le (P_0{L}^2{)}^{1\mathrm{/}2}$ therefore give $\mathbb{E}{\sup }_{\mathrm{\parallel }t\mathrm{\parallel }\le r}\mathrm{\mid }{\mathbb{G}}_n(f_t-f_0)\mathrm{\mid }\le C_{d}r\mathrm{\parallel }L{\mathrm{\parallel }}_2\mathrm{.}$ For $r_t=f_t-t^T{s}_{{\theta }_0}$, step 2.1 gives ${\sup }_{\mathrm{\parallel }t\mathrm{\parallel }\le M\mathrm{/}\sqrt{n}}{P}_0{r}_t^2=o(n^{-1})$, and increments have the square-integrable pointwise envelope $L+\mathrm{\parallel }{s}_{{\theta }_0}\mathrm{\parallel }$. Fix a mesh $\epsilon >0$ and cover the bounded $h$-ball by finitely many mesh-$\epsilon$ points. At each center $h_j$, the variance of $\sqrt{n}{\mathbb{G}}_n{r}_{h_j\mathrm{/}\sqrt{n}}$ is $n{\mathrm{Var}}_0(r_{h_j\mathrm{/}\sqrt{n}})\le n{P}_0{r}_{h_j\mathrm{/}\sqrt{n}}^2=o(1)$, so Chebyshev and finiteness of the net make the maximum over centers $o_P(1)$. For increments within cells, form $g_h=r_{h\mathrm{/}\sqrt{n}}-r_{h_j\mathrm{/}\sqrt{n}}$, where $h_j$ is the center of the cell containing $h$. Each $g_h$ is pointwise bounded by $\epsilon (L+\mathrm{\parallel }{s}_{{\theta }_0}\mathrm{\parallel })\mathrm{/}\sqrt{n}$. Apply conditional Rademacher chaining to the union of cell classes: at dyadic level $k$ there are at most $C_M{\epsilon }^{-d}{2}^{kd}$ links, each with conditional scale at most $C\epsilon 2^{-k}(P_n(L+\mathrm{\parallel }{s}_{{\theta }_0}\mathrm{\parallel }{)}^2{)}^{1\mathrm{/}2}$ after the $\sqrt{n}$ normalization. The exponential maximum bound and summation of ${\sum }_{k\ge 0}{2}^{-k}\sqrt{\log (C_M{\epsilon }^{-d})+C_{d}k}$ give an expected supremum at most $C_M\epsilon \sqrt{\log (C_M\mathrm{/}\epsilon )}$ after symmetrization and $\mathbb{E}(P_n(L+\mathrm{\parallel }{s}_{{\theta }_0}\mathrm{\parallel }{)}^2{)}^{1\mathrm{/}2}\le \mathrm{\parallel }L+\mathrm{\parallel }{s}_{{\theta }_0}\mathrm{\parallel }{\mathrm{\parallel }}_2$. Markov's inequality then shows that these cell increments vanish in probability as $\epsilon \downarrow 0$. Letting $n\to \mathrm{\infty }$ and then $\epsilon \downarrow 0$ proves ${\sup }_{\mathrm{\parallel }h\mathrm{\parallel }\le M}\mathrm{\mid }\sqrt{n}{\mathbb{G}}_n{r}_{h\mathrm{/}\sqrt{n}}\mathrm{\mid }=o_P(1)$. Thus, for each fixed $M$, uniformly over $\mathrm{\parallel }h\mathrm{\parallel }\le M$, ${\mathrm{\ell }}_n({\theta }_0+h\mathrm{/}\sqrt{n})-{\mathrm{\ell }}_n({\theta }_0)=h^T{\mathrm{\Delta }}_n-\frac{1}{2}{h}^{T}Ih+o_{P_0}(1),{\mathrm{\Delta }}_n=n^{-1\mathrm{/}2}{\sum }_{i=1}^n{s}_{{\theta }_0}(X_i)\mathrm{.}$ The expectation term is step 2.1; the centered term follows by writing $f_t=t^T{s}_{{\theta }_0}+(f_t-t^T{s}_{{\theta }_0})$ and using the two net bounds.

Reduce ${\delta }_0$ if needed so the log envelope holds on the radius $2{\delta }_0$ ball, and put $\delta ={\delta }_0\mathrm{/}2$. Consistency makes ${\widehat{\theta }}_n$ lie within $\delta$ of ${\theta }_0$ with probability tending to one. On the dyadic shells $r_j\le \mathrm{\parallel }t\mathrm{\parallel }\le 2r_j$ with $r_j=2^{j}M\mathrm{/}\sqrt{n}$, exact maximization and ${\mathrm{\ell }}_n({\widehat{\theta }}_n)\ge {\mathrm{\ell }}_n({\theta }_0)$ require $\sqrt{n}{\sup }_{\mathrm{\parallel }t\mathrm{\parallel }\le 2r_j}\mathrm{\mid }{\mathbb{G}}_n{f}_t\mathrm{\mid }\ge cn{r}_j^2$. The expectation bound in step 3.1 and Markov's inequality [F11] bound the probability of this event by $C\mathrm{/}(\sqrt{n}{r}_j)=C\mathrm{/}(2^{j}M)$. Summing the geometric series and then letting $M\to \mathrm{\infty }$ proves $\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)=O_{P_0}(1)$ using [F4]. The argument used only consistency, feasibility of ${\theta }_0$, and the maximizing inequality; it therefore proves the same bound for any exact maximizer over a subset containing ${\theta }_0$.

Put ${\widehat{h}}_n=\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)$ and $h_n^{\ast }=I^{-1}{\mathrm{\Delta }}_n$. The multivariate CLT [F7] and [F5] make $h_n^{\ast }=O_{P_0}(1)$; step 4.1 makes ${\widehat{h}}_n=O_{P_0}(1)$. For $Q_n(h)=h^T{\mathrm{\Delta }}_n-\frac{1}{2}{h}^{T}Ih$, $Q_n(h_n^{\ast })-Q_n(h)=\frac{1}{2}(h-h_n^{\ast }{)}^{T}I(h-h_n^{\ast })\ge \frac{1}{2}{\lambda }_{\min }(I)\mathrm{\parallel }h-h_n^{\ast }{\mathrm{\parallel }}^2\mathrm{.}$ Uniform convergence on bounded balls in step 3.1, feasibility of bounded local $h$ because ${\theta }_0$ is interior, and exact MLE maximization therefore force ${\widehat{h}}_n-h_n^{\ast }=o_{P_0}(1)$.

The CLT [F7] gives ${\mathrm{\Delta }}_n\Rightarrow N_d(0,I)$; the limiting law exists by [F8]. Since $I$ is invertible, $h_n^{\ast }=I^{-1}{\mathrm{\Delta }}_n\Rightarrow N_d(0,I^{-1})$ by the continuous linear map. Combining this with step 5.1 by [F6] proves both the asymptotic linear representation and the stated normal limit. AC is used only through [F7]–[F8].

If $d=q$, then $I_{\mathrm{e}\mathrm{f}\mathrm{f}}=I$ is positive definite. If $d>q$, $I_{\eta \eta }$ is a positive-definite principal block. For nonzero $v\in {\mathbb{R}}^q$, evaluate the quadratic form of $I$ at $(v,-I_{\eta \eta }^{-1}{I}_{\eta \psi }v)$; the result is $v^T{I}_{\mathrm{e}\mathrm{f}\mathrm{f}}v>0$, so $I_{\mathrm{e}\mathrm{f}\mathrm{f}}$ is positive definite in either case. Normalizing $p_{{\theta }_0+h}$ in the QMD expansion [F1], write $\sqrt{p_{{\theta }_0+h}}=\sqrt{p_{{\theta }_0}}+\frac{1}{2}{h}^T{s}_{{\theta }_0}\sqrt{p_{{\theta }_0}}+e_h$ with $\mathrm{\parallel }{e}_h{\mathrm{\parallel }}_{L^2(\mu )}=o(\mathrm{\parallel }h\mathrm{\parallel })$. Expanding the squared norm, whose value is $\int p_{{\theta }_0+h}d\mu =1$, gives $h^T{\mathbb{E}}_{{\theta }_0}{s}_{{\theta }_0}+O(\mathrm{\parallel }h{\mathrm{\parallel }}^2)+o(\mathrm{\parallel }h\mathrm{\parallel })=0$. Taking $h=t{e}_j$ and dividing by $t$ proves each score coordinate has mean zero. Since $s_{{\theta }_0}$ is square integrable by [F1], the iid CLT [F7] gives ${\mathrm{\Delta }}_n\Rightarrow N_d(0,I)$ under AC. Define $r_n={\mathrm{\Delta }}_{\psi ,n}-I_{\psi \eta }{I}_{\eta \eta }^{-1}{\mathrm{\Delta }}_{\eta ,n}$. Its covariance, by block multiplication, is $\mathrm{Var}(r_n)=I_{\psi \psi }-I_{\psi \eta }{I}_{\eta \eta }^{-1}{I}_{\eta \psi }-I_{\psi \eta }{I}_{\eta \eta }^{-1}{I}_{\eta \psi }+I_{\psi \eta }{I}_{\eta \eta }^{-1}{I}_{\eta \eta }{I}_{\eta \eta }^{-1}{I}_{\eta \psi }=I_{\mathrm{e}\mathrm{f}\mathrm{f}}\mathrm{.}$ Thus the linear-map case of Slutsky [F6] gives $r_n\Rightarrow N_q(0,I_{\mathrm{e}\mathrm{f}\mathrm{f}})$; its covariance is the block calculation above.

By [F3], the full MLE satisfies $\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)=I^{-1}{\mathrm{\Delta }}_n+o_P(1)$. Solving the block system $I(a,b{)}^T=({\mathrm{\Delta }}_{\psi ,n},{\mathrm{\Delta }}_{\eta ,n}{)}^T$ shows its tested block is $a=I_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}{r}_n$, including the empty-nuisance case. If $d=q$, the local null contains only ${\theta }_0$, so consistency makes the constrained MLE equal to ${\theta }_0$ with probability tending to one and there is no nuisance expansion to prove. Otherwise, [F3] makes the constrained root-$n$ error bounded. In local coordinates $h=(u,v)$ for $(\eta ,\psi )$, the limiting quadratic log likelihood is $Q_n(u,v)=u^T{\mathrm{\Delta }}_{\eta ,n}+v^T{\mathrm{\Delta }}_{\psi ,n}-\frac{1}{2}\{u^T{I}_{\eta \eta }u+2u^T{I}_{\eta \psi }v+v^T{I}_{\psi \psi }v\}\mathrm{.}$ For general $v$, completing the square in $u$ gives the unique maximizer $u=I_{\eta \eta }^{-1}({\mathrm{\Delta }}_{\eta ,n}-I_{\eta \psi }v)$ and the profile $v^T{r}_n-\frac{1}{2}{v}^T{I}_{\mathrm{e}\mathrm{f}\mathrm{f}}v$ up to a term independent of $v$. At the null $v=0$ this gives the unique nuisance maximizer $u_0=I_{\eta \eta }^{-1}{\mathrm{\Delta }}_{\eta ,n}$. On every distance-$\epsilon$ set from $u_0$, strict positive definiteness of $I_{\eta \eta }$ makes the quadratic log likelihood at least $\frac{1}{2}{\lambda }_{\min }(I_{\eta \eta }){\epsilon }^2$ below its maximum. Uniformity of the likelihood expansion [A1], constrained root-$n$ tightness [F3], and this gap therefore give $\sqrt{n}({\tilde{\eta }}_n-{\eta }_0)=I_{\eta \eta }^{-1}{\mathrm{\Delta }}_{\eta ,n}+o_P(1)$.

Consistency places both maximizers in the open chart neighborhood with probability tending to one; the off-chart zero conventions in [F2] therefore do not affect the expansions below. The upper-left block of $I^{-1}$ is $I_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}$ by solving the block system $I(x,y{)}^T=(a,0{)}^T$. The differentiable local chart has tested coordinate $\psi$, so the Wald statistic in [F2], the expansion in step 2.1, and covariance consistency give $W_n=(I_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}{r}_n{)}^T{I}_{\mathrm{e}\mathrm{f}\mathrm{f}}(I_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}{r}_n)+o_P(1)=r_n^T{I}_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}{r}_n+o_P(1)\mathrm{.}$

If $d=q$, the nuisance score is empty and the score expansion [A1] at $h=0$ gives $U_{\mathrm{e}\mathrm{f}\mathrm{f},n}=U_{\psi ,n}({\theta }_0)={\mathrm{\Delta }}_n+o_P(1)=r_n+o_P(1)$. If $d>q$, the score expansion [A1] at the constrained MLE and $\sqrt{n}({\tilde{\eta }}_n-{\eta }_0)=I_{\eta \eta }^{-1}{\mathrm{\Delta }}_{\eta ,n}+o_P(1)$ give $U_{\psi ,n}({\tilde{\theta }}_n)=r_n+o_P(1)$ and $U_{\eta ,n}({\tilde{\theta }}_n)=o_P(1)$. The estimated projection coefficient is bounded in probability by consistency, so the efficient score in [F2] is $r_n+o_P(1)$. Efficient-information consistency and [F5] imply $S_n=r_n^T{I}_{\mathrm{e}\mathrm{f}\mathrm{f}}^{-1}{r}_n+o_P(1)$.