Semiparametric Efficiency

Put $Z_n=n^{-1\mathrm{/}2}{\sum }_{i=1}^n\phi (X_i)$ and $R_n=\sqrt{n}\{{\widehat{\psi }}_n-\psi (P)\}-Z_n$. By [F1], $R_n\to 0$ in probability. The vectors $\phi (X_i)$ are iid, centered, and have finite second moment, so [F2] gives $Z_n\Rightarrow Z\sim N_k(0,{\mathrm{\Sigma }}_{\phi })$. The covariance is positive semidefinite by $a^T{\mathrm{\Sigma }}_{\phi }a=P[(a^T\phi {)}^2]\ge 0$ for every $a\in {\mathbb{R}}^k$.

Fix any $a\in {\mathbb{R}}^k$. By [F3], $a^T{Z}_n\Rightarrow a^{T}Z$. Also $\mathrm{\mid }{a}^T{R}_n\mathrm{\mid }\le \mathrm{\parallel }a\mathrm{\parallel }\mathrm{\parallel }{R}_n\mathrm{\parallel }$, so $a^T{R}_n\to 0$ in probability. Apply [F4] to obtain $a^T(Z_n+R_n)\Rightarrow a^{T}Z$.

Step 2.1 holds for every $a$, and $Z$ is the specified Borel Gaussian law from step 1.1. [F5] therefore gives $Z_n+R_n\Rightarrow Z$, which is the asserted limit. No nonsingularity of ${\mathrm{\Sigma }}_{\phi }$ is used.

Under $P_{n,0}$, $T_n\Rightarrow W$ by regularity at $h=0$ and ${\mathrm{\Delta }}_n\Rightarrow N_d(0,K)$ by LAN. Marginal tightness gives compact sets with outside probabilities below $\epsilon \mathrm{/}2$; enlarge them to closed balls. Their product is closed and bounded in ${\mathbb{R}}^{k+d}$ and hence compact by [F11]. By [F12] the joint outside probability is at most $\epsilon$, so the joint laws of $(T_n,{\mathrm{\Delta }}_n)$ are tight.

By [F4], choose a subsequence along which $(T_n,{\mathrm{\Delta }}_n)\Rightarrow (T,\mathrm{\Delta })$. Its marginals have laws $W$ and $N_d(0,K)$. Fix $h\in {\mathbb{R}}^d$. By [F1], $P_{n,0}(L_{n,h}=0)\to 0$; define ${\mathrm{\ell }}_{n,h}=\log L_{n,h}$ when $L_{n,h}>0$ and ${\mathrm{\ell }}_{n,h}=0$ otherwise. This replacement changes a variable only on an event of probability $o(1)$. For any $a\in {\mathbb{R}}^k$ and $b\in \mathbb{R}$, the projection of $(T_n,{\mathrm{\ell }}_{n,h})$ differs by $o_P(1)$ from $a^T{T}_n+b{h}^T{\mathrm{\Delta }}_n-\frac{1}{2}b{h}^{T}Kh$. The latter converges by [F7]; applying scalar Slutsky and [F9] to every projection gives $(T_n,{\mathrm{\ell }}_{n,h})\Rightarrow (T,h^T\mathrm{\Delta }-\frac{1}{2}{h}^{T}Kh)$.

By the continuous mapping theorem [F7], $(T_n,e^{{\mathrm{\ell }}_{n,h}})\Rightarrow (T,e^{h^T\mathrm{\Delta }-\frac{1}{2}{h}^{T}Kh})$. Since $e^{{\mathrm{\ell }}_{n,h}}=L_{n,h}$ off the event $L_{n,h}=0$, whose probability tends to zero by [F1], the same joint limit holds with $L_{n,h}$. Apply [F3] to this mean-one proxy; [F1] transfers its limit under the proxy law to the actual $P_{n,h}$ law. Regularity and ${\delta }_n(h)\to Dh$ imply that under this same actual law $T_n\Rightarrow W+Dh$: apply [F7] to each real projection and then [F9]. Therefore for every real $h$ and $t\in {\mathbb{R}}^k$, $E[e^{i{t}^{T}T+h^T\mathrm{\Delta }-\frac{1}{2}{h}^{T}Kh}]=e^{i{t}^{T}Dh}{\phi }_W(t),$ where ${\phi }_W(t)=E[e^{i{t}^{T}W}]$.

Rearranging step 3.1 gives $F_t(h):=E[e^{i{t}^{T}T+h^T\mathrm{\Delta }}]=e^{\frac{1}{2}{h}^{T}Kh+i{t}^{T}Dh}{\phi }_W(t)$ for every real $h$. For complex $z\in {\mathbb{C}}^d$, define $F_t(z)=E[e^{i{t}^{T}T+z^T\mathrm{\Delta }}]$. On every compact set of $z$, its coordinate power series and derivatives are dominated by $e^{R\mathrm{\parallel }\mathrm{\Delta }{\mathrm{\parallel }}_1}$ for some finite $R$, whose expectation is finite because $\mathrm{\Delta }$ is Gaussian. Thus $F_t$ is entire in each coordinate, as is the right-hand side. Apply [F5] one coordinate at a time, holding the remaining coordinates real and then extending them successively; the equality holds for every complex $z$.

Put $z=iu$ in step 4.1. By [F13], the resulting characteristic functions satisfy, for all real $t,u$, $E[e^{i{t}^{T}T+i{u}^T\mathrm{\Delta }}]={\phi }_W(t)e^{-t^{T}Du-u^{T}Ku\mathrm{/}2}$. Set $R=T-D{K}^{-1}\mathrm{\Delta }$ and replace $u$ by $u-K^{-1}{D}^{T}t$. Direct expansion gives $E[e^{i{t}^{T}R+i{u}^T\mathrm{\Delta }}]=({\phi }_W(t)e^{t^{T}D{K}^{-1}{D}^{T}t\mathrm{/}2})e^{-u^{T}Ku\mathrm{/}2}$. At $u=0$ the first factor is the characteristic function of $R$; the second is the characteristic function of $\mathrm{\Delta }$. The joint characteristic function therefore factors as the product of the marginal characteristic functions.

Let $\rho =\mathcal{L}(R)\otimes N_d(0,K)$, whose product law exists by [F10]. For each $t\in {\mathbb{R}}^k$ and $u\in {\mathbb{R}}^d$, the function $(r,z)\mapsto e^{i{t}^{T}r}{e}^{i{u}^{T}z}$ is bounded and hence integrable under $\rho$; [F14] factors its integral into the marginal characteristic functions. By [F13] and step 5.1, this product equals the characteristic function of $(R,\mathrm{\Delta })$. Scalar uniqueness [F8] and [F9] give $\mathcal{L}(R,\mathrm{\Delta })=\rho$, so $R$ and $\mathrm{\Delta }$ are independent. Since $D{K}^{-1}\mathrm{\Delta }\sim N_k(0,D{K}^{-1}{D}^T)$ and $T=D{K}^{-1}\mathrm{\Delta }+R$, the law of $T$, which is $W$, is the stated convolution. The argument allows singular $D{K}^{-1}{D}^T$ and uses no inverse of that matrix.

For $t>0$ let $r_t=\sqrt{p_t}-\sqrt{p_0}-\frac{t}{2}s\sqrt{p_0}$. On $S^c$, $r_t=\sqrt{p_t}$, so ${\alpha }_t={\int }_{S^c}{p}_{t}d\mu \le \mathrm{\parallel }{r}_t{\mathrm{\parallel }}_2^2=o(t^2)$. Let $B_t=S\cap \{p_t=0\}$. Then $P(B_t)\le \mathrm{\parallel }\sqrt{p_t}-\sqrt{p_0}{\mathrm{\parallel }}_2^2=O(t^2)$. On $B_t$, the DQM identity gives $\sqrt{p_0}=-\frac{t}{2}s\sqrt{p_0}-r_t$, hence $\sqrt{P(B_t)}\le \frac{t}{2}\{P[s^2{\mathbf{1}}_{B_t}]{\}}^{1\mathrm{/}2}+o(t)$. Since $P(B_t)\to 0$ and $s^2$ is integrable, truncating $s^2$ at a fixed bound and then increasing that bound shows $P[s^2{\mathbf{1}}_{B_t}]\to 0$. Thus ${\beta }_t:=P(p_t=0)=P(B_t)=o(t^2)$.

For small $t$, $1-{\alpha }_t>0$. The definition of $q_t$ gives $P{q}_t=(1-{\alpha }_t{)}^{-1}{\int }_S{p}_{t}d\mu =1$. On $S$, divide the DQM expansion by $\sqrt{p_0}$ and include the normalizer $(1-{\alpha }_t{)}^{-1\mathrm{/}2}=1+o(t^2)$. With $Y_t=2(\sqrt{q_t}-1)$, this yields $Y_t=ts+R_t$ in $L^2(P)$ with $\mathrm{\parallel }{R}_t{\mathrm{\parallel }}_2=o(t)$.

Since $P{q}_t=1$ and $q_t=(1+Y_t\mathrm{/}2{)}^2$ $P$-almost surely, linearity [F10] expands this normalization identity to $P{Y}_t=-\frac{1}{4}P[Y_t^2]$. By [F2], [F3], and step 2.1, $P[Y_t^2]=t^{2}I+o(t^2)$, so $P{Y}_t=-\frac{1}{4}{t}^{2}I+o(t^2)$.

Suppose $h>0$. By [F4] applied to $s^2$ and [F5] applied to $s(X_i)$, $n^{-1}{\sum }_{i}s(X_i{)}^2\to I$ in probability and ${\mathrm{\Delta }}_n\Rightarrow N(0,I)$. Step 2.1 and [F3] give $E\mathrm{\mid }{Y}_{t_n}^2-t_n^2{s}^2\mathrm{\mid }\le \mathrm{\parallel }{R}_{t_n}{\mathrm{\parallel }}_2(\mathrm{\parallel }{Y}_{t_n}{\mathrm{\parallel }}_2+t_n\mathrm{\parallel }s{\mathrm{\parallel }}_2)=o(t_n^2)$. Since $n{t}_n^2=h^2$, linearity [F10] and identical distribution [F6] show that the expected sum of these absolute differences is $o(1)$, so Markov's inequality [F8] makes that sum converge to zero in probability. Also $t_n^2{\sum }_{i}s(X_i{)}^2=(h^2\mathrm{/}n){\sum }_{i}s(X_i{)}^2\to h^{2}I$ in probability; therefore ${\sum }_i{Y}_{t_n}(X_i{)}^2\to h^{2}I$.

The centered iid sum of $R_{t_n}(X_i)$ has variance at most $n\mathrm{\parallel }{R}_{t_n}{\mathrm{\parallel }}_2^2=o(n{t}_n^2)=o(1)$, so [F8] (applied to its square) gives ${\sum }_i{R}_{t_n}(X_i)=nP{R}_{t_n}+o_P(1)$. By [F2], $P{R}_{t_n}=P{Y}_{t_n}$; step 3.1 therefore gives ${\sum }_i{Y}_{t_n}(X_i)=h{\mathrm{\Delta }}_n-\frac{1}{4}{h}^{2}I+o_P(1)$.

The variables $V_n=\sqrt{n}{Y}_{t_n}(X_1)$ converge in $L^2(P)$ to $hs(X_1)$ by step 2.1. By [F3], $E\mathrm{\mid }{V}_n^2-h^{2}s(X_1{)}^2\mathrm{\mid }\le \mathrm{\parallel }{V}_n-hs(X_1){\mathrm{\parallel }}_2(\mathrm{\parallel }{V}_n{\mathrm{\parallel }}_2+h\mathrm{\parallel }s{\mathrm{\parallel }}_2)\to 0$, since the second factor is bounded. Thus [F12] makes $V_n^2$ uniformly integrable. For every $\eta >0$, [F8] and [F9] give $P^{\otimes n}({\max }_{i\le n}\mathrm{\mid }{Y}_{t_n}(X_i)\mathrm{\mid }>\eta )\le {\eta }^{-2}E[V_n^2{\mathbf{1}}_{\{\mathrm{\mid }{V}_n\mathrm{\mid }>\eta \sqrt{n}\}}]\to 0$, because the tail threshold for $V_n^2$ tends to infinity. Also $P^{\otimes n}(L_n=0)\le n{\beta }_{t_n}=o(1)$ by step 1.1. On the event all factors are positive and ${\max }_i\mathrm{\mid }{Y}_{t_n}(X_i)\mathrm{\mid }\le 1$, let $g(y)=2\log (1+y\mathrm{/}2)$; its third derivative is $g^{\mathrm{\prime }\mathrm{\prime }\mathrm{\prime }}(y)=\frac{1}{2}(1+y\mathrm{/}2{)}^{-3}$, so $\mathrm{\mid }{g}^{\mathrm{\prime }\mathrm{\prime }\mathrm{\prime }}(y)\mathrm{\mid }\le 4$ on $[-1,1]$. [F11] gives $g(y)=y-\frac{1}{4}{y}^2+O(\mathrm{\mid }y{\mathrm{\mid }}^3)$ there, with a uniform constant. Thus $\log q_{t_n}(X_i)=Y_{t_n}(X_i)-\frac{1}{4}{Y}_{t_n}(X_i{)}^2+O(\mathrm{\mid }{Y}_{t_n}(X_i){\mathrm{\mid }}^3)$. The sum of cubic terms is bounded by a constant times ${\max }_i\mathrm{\mid }{Y}_{t_n}(X_i)\mathrm{\mid }{\sum }_i{Y}_{t_n}(X_i{)}^2=o_P(1)$. Steps 3.2–4.1 now yield $\log L_n=h{\mathrm{\Delta }}_n-\frac{1}{2}{h}^{2}I+o_P(1)$.

Decompose $P_{t_n}=(1-{\alpha }_{t_n}){\bar{P}}_{t_n}+{\alpha }_{t_n}{S}_{t_n}$, where ${\bar{P}}_{t_n}=q_{t_n}P$ and, when ${\alpha }_{t_n}>0$, $S_{t_n}$ is the normalized restriction to $S^c$. Expanding the product mixture, the all-${\bar{P}}_{t_n}$ term has weight $(1-{\alpha }_{t_n}{)}^n$ and law ${\bar{P}}_{t_n}^{\otimes n}$. If ${\alpha }_{t_n}>0$, normalize the remaining mixture terms to a probability law $R_{n,t_n}$; then $P_{t_n}^{\otimes n}=(1-{\alpha }_{t_n}{)}^n{\bar{P}}_{t_n}^{\otimes n}+(1-(1-{\alpha }_{t_n}{)}^n)R_{n,t_n}$. When ${\alpha }_{t_n}=0$, the two product laws are equal. Otherwise, this mixture identity and the event-supremum definition [F7] give $\mathrm{\parallel }{P}_{t_n}^{\otimes n}-{\bar{P}}_{t_n}^{\otimes n}{\mathrm{\parallel }}_{\mathrm{T}\mathrm{V}}\le 1-(1-{\alpha }_{t_n}{)}^n\le n{\alpha }_{t_n}$ by finite subadditivity [F9]. Since ${\alpha }_{t_n}=o(t_n^2)$ and $n{t}_n^2=h^2$ for fixed $h>0$, this is $o(1)$. For the asserted product-density identity, the coordinate factors $q_{t_n}$ are nonnegative measurable, so [F14] makes $L_n(x_1,\dots ,x_n)={\prod }_i{q}_{t_n}(x_i)$ product-measurable. Define the measure $Q_n(A)={\int }_A{L}_{n}d{P}^{\otimes n}$ by [F13]. Repeated Tonelli [F16] on a measurable rectangle $A_1\times \cdots \times A_n$ gives $Q_n(A_1\times \cdots \times A_n)={\prod }_{i=1}^n{\int }_{A_i}{q}_{t_n}dP={\prod }_{i=1}^n{\bar{P}}_{t_n}(A_i)$, where the one-coordinate identity is the definition ${\bar{P}}_{t_n}=q_{t_n}P$. In particular, step 2.1 gives $Q_n({\mathcal{X}}^n)=(P{q}_{t_n}{)}^n=1$. By [F15], ${\bar{P}}_{t_n}^{\otimes n}$ is also a probability and has the same rectangle values. The measurable rectangles form a pi-system (their intersections are coordinate intersections) generating the finite product sigma-algebra by [F14]; the constant exhaustion ${\mathcal{X}}^n$ belongs to it and both measures have mass one there. Thus [F17] proves $Q_n={\bar{P}}_{t_n}^{\otimes n}$ on every product-measurable event. Consequently $L_n{P}^{\otimes n}={\bar{P}}_{t_n}^{\otimes n}$, identifying exactly the proxy in the statement with the product law compared above. If $h=0$, both product laws equal $P^{\otimes n}$ and $L_n=1$. This proves every assertion.

Fix an allowed path with score $s$ and a fixed $h>0$. Let $T_n=\sqrt{n}\{{\widehat{\psi }}_n-\psi (P)\}$, ${\mathrm{\Delta }}_n=n^{-1\mathrm{/}2}{\sum }_{i}s(X_i)$, and $L_n$ be the proxy ratio from [F3]. By [F3], $P(L_n=0)\to 0$; set ${\mathrm{\ell }}_n=\log L_n$ when $L_n>0$ and ${\mathrm{\ell }}_n=0$ otherwise. For arbitrary $a\in {\mathbb{R}}^k$ and $b\in \mathbb{R}$, asymptotic linearity and [F3] give $a^T{T}_n+b{\mathrm{\ell }}_n=n^{-1\mathrm{/}2}{\sum }_{i=1}^n\{a^T\phi (X_i)+bhs(X_i)\}-\frac{1}{2}b{h}^{2}P[s^2]+o_P(1)\mathrm{.}$ The summands are iid and square-integrable by [F1] and [F2].

Apply [F5] to each scalar linear combination in step 1.1, then apply [F6] to its $o_P(1)$ remainder. Since this holds for every $(a,b)$, [F7] gives the joint Gaussian limit of $(T_n,{\mathrm{\ell }}_n)$, with first marginal $N_k(0,{\mathrm{\Sigma }}_{\phi })$ and cross-covariance between $T_n$ and ${\mathrm{\ell }}_n$ equal to $hP[\phi s]$.

The log-ratio marginal in step 2.1 is $N(-h^{2}P[s^2]\mathrm{/}2,h^{2}P[s^2])$. Exponentiating the finite version ${\mathrm{\ell }}_n$ by [F11] gives the joint likelihood-ratio limit, and replacing $e^{{\mathrm{\ell }}_n}$ by $L_n$ changes it only on the $o(1)$-probability zero event from step 1.1. Apply [F4] to the proxy law in [F3]; under that law, $T_n\Rightarrow N_k(hP[\phi s],{\mathrm{\Sigma }}_{\phi })$. Total-variation closeness in [F3] transfers the same limit to the actual law $P_{h\mathrm{/}\sqrt{n}}^{\otimes n}$.

By pathwise differentiability, $\sqrt{n}\{\psi (P_{h\mathrm{/}\sqrt{n}})-\psi (P)\}\to h{\dot{\psi }}_P(s)$. The regularity hypothesis therefore makes $T_n$ under the actual local law converge to $N_k(h{\dot{\psi }}_P(s),{\mathrm{\Sigma }}_{\phi })$, using the base limit [F8]. Comparing with step 3.1, uniqueness of weak limits gives $N_k(hP[\phi s],{\mathrm{\Sigma }}_{\phi })=N_k(h{\dot{\psi }}_P(s),{\mathrm{\Sigma }}_{\phi })$. Equal-covariance Gaussian laws have equal means, so $P[\phi s]={\dot{\psi }}_P(s)$.

Step 4.1 holds for every allowed score. By linearity it holds on their linear span; by [F10] the pairing with $\phi$ is continuous, and by [F2] the derivative is continuous, so equality extends to the closure $\mathcal{T}$. Thus $\phi$ is an ambient representer of the pathwise derivative.

By [F9], $r=\phi -{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}$ has every coordinate in ${\mathcal{T}}^{\perp }$. Hence $P[{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}{r}^T]=0=P[r{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}^T]$, and expansion gives $P[\phi {\phi }^T]-V_{\mathrm{c}\mathrm{a}\mathrm{n}}=P[r{r}^T]$. For every $a\in {\mathbb{R}}^k$, $a^{T}P[r{r}^T]a=P[(a^{T}r{)}^2]\ge 0$, proving positive-semidefinite dominance.

For any $a\in {\mathbb{R}}^k$, the projection of $a^T{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}$ onto $\mathcal{S}$ is $a^{T}D{I}^{-1}S$: its residual has inner product $a^{T}D-a^{T}D{I}^{-1}P[S{S}^T]=0$ with every linear combination of the score coordinates. Therefore the covariance matrix of the projected gradient is $D{I}^{-1}{D}^T$.

By [F2] and Quadratic-mean differentiability implies iid LAN, this finite-dimensional submodel has the required LAN proxy and total-variation control. The regularity premise in the statement and [F4] give , where $G_{\gamma }\sim N_k(0,D{I}^{-1}{D}^T)$ is independent of $W_{\gamma }$. Since $L$ has finite covariance and $G_{\gamma }$ does, $W_{\gamma }$ has finite covariance as well; independence yields $\mathrm{Cov}(L)=D{I}^{-1}{D}^T+\mathrm{Cov}(W_{\gamma })⪰D{I}^{-1}{D}^T$.

If $\mathcal{T}=\{0\}$, then ${\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}=0$ and $\mathrm{Cov}(L)⪰0=V_{\mathrm{c}\mathrm{a}\mathrm{n}}$. Otherwise fix $a\in {\mathbb{R}}^k$ and $\epsilon >0$. Density of the union of the ${\mathcal{S}}_{\gamma }$ in $\mathcal{T}$ provides an index $\gamma$ and $g\in {\mathcal{S}}_{\gamma }$ with $\mathrm{\parallel }g-a^T{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}{\mathrm{\parallel }}_{L^2(P)}<\epsilon$. The projection formula in step 1.1 is orthogonal, so for any $g\in {\mathcal{S}}_{\gamma }$, $\mathrm{\parallel }{a}^T{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}-g{\mathrm{\parallel }}_2^2=\mathrm{\parallel }{a}^T{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}-{\mathrm{\Pi }}_{{\mathcal{S}}_{\gamma }}(a^T{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}}){\mathrm{\parallel }}_2^2+\mathrm{\parallel }{\mathrm{\Pi }}_{{\mathcal{S}}_{\gamma }}(a^T{\varphi }_{\mathrm{c}\mathrm{a}\mathrm{n}})-g{\mathrm{\parallel }}_2^2$. Thus the projection error is below $\epsilon$, and the quadratic forms $a^T{D}_{\gamma }{I}_{\gamma }^{-1}{D}_{\gamma }^{T}a$ approach $a^T{V}_{\mathrm{c}\mathrm{a}\mathrm{n}}a$ as $\epsilon \downarrow 0$.

When $\mathcal{T}\mathrm{\ne }\{0\}$, step 2.1 gives $a^T\mathrm{Cov}(L)a\ge a^T{D}_{\gamma }{I}_{\gamma }^{-1}{D}_{\gamma }^{T}a$ for every $\gamma$. Take the approximation in step 2.2 with arbitrarily small $\epsilon$ to obtain $a^T\mathrm{Cov}(L)a\ge a^T{V}_{\mathrm{c}\mathrm{a}\mathrm{n}}a$. Since this holds for every $a$, $\mathrm{Cov}(L)⪰V_{\mathrm{c}\mathrm{a}\mathrm{n}}$. Together with the zero-tangent case in step 2.2, this proves the claim.