Delta Method and Asymptotic Normality

By [F6], $Z_n\Rightarrow Z$ is weak convergence in ${\mathbb{R}}^d$. By [F2], this implies $Z_n=O_P(1)$.

Since $r_n^{-1}\to 0$, the deterministic sequence $r_n^{-1}$ is $o_P(1)$. Applying the bilinear product rule in [F3] to scalar multiplication gives $T_n-\theta =r_n^{-1}{Z}_n=o_P(1)$.

Fix $\epsilon ,\eta >0$. By [F2] choose $M>0$ with ${\mathrm{lim\hspace{0.17em}sup}}_{n}P(\mathrm{\parallel }{Z}_n\mathrm{\parallel }>M)\le \eta$. The first-order remainder limit in the statement gives $\delta >0$ such that $\mathrm{\parallel }{\rho }_1(h)\mathrm{\parallel }\le (\epsilon \mathrm{/}M)\mathrm{\parallel }h\mathrm{\parallel }$ when $0<\mathrm{\parallel }h\mathrm{\parallel }<\delta$. Step 2.1 gives $P(\mathrm{\parallel }{T}_n-\theta \mathrm{\parallel }\ge \delta )\to 0$, so the limiting upper probability that $r_n\mathrm{\parallel }{\rho }_1(T_n-\theta )\mathrm{\parallel }>\epsilon$ is at most $\eta$. Letting $\eta \downarrow 0$ proves $r_n{\rho }_1(T_n-\theta )=o_P(1)$. Substitution in the first expansion yields $r_n\{g(T_n)-g(\theta )\}=A{Z}_n+o_P(1)$.

Fix $\epsilon ,\eta >0$. By [F2] choose $M>0$ with ${\mathrm{lim\hspace{0.17em}sup}}_{n}P(\mathrm{\parallel }{Z}_n\mathrm{\parallel }>M)\le \eta$. The second-order remainder limit in the statement gives $\delta >0$ such that $\mathrm{\parallel }{\rho }_2(h)\mathrm{\parallel }\le (\epsilon \mathrm{/}{M}^2)\mathrm{\parallel }h{\mathrm{\parallel }}^2$ for $0<\mathrm{\parallel }h\mathrm{\parallel }<\delta$. Step 2.1 puts $T_n-\theta$ inside this neighborhood with probability tending to one. Outside the two tail events, $r_n^2\mathrm{\parallel }{\rho }_2(T_n-\theta )\mathrm{\parallel }\le \epsilon$ because $r_n\mathrm{\parallel }{T}_n-\theta \mathrm{\parallel }=\mathrm{\parallel }{Z}_n\mathrm{\parallel }$. Thus $r_n^2{\rho }_2(T_n-\theta )=o_P(1)$; since $A=0$, substitution in the second expansion gives $r_n^2\{g(T_n)-g(\theta )\}=\frac{1}{2}H[Z_n,Z_n]+o_P(1)$.

The map $(x,e)\mapsto Ax+e$ is continuous. Apply [F4] to $Z_n\Rightarrow Z$ and the $o_P(1)$ error in step 3.1 to obtain the first-order limit $AZ$.

The quadratic map $z\mapsto \frac{1}{2}H[z,z]$ is continuous. Apply [F4] to its value at $Z_n$ and the error in step 3.2 to obtain the second-order limit $\frac{1}{2}H[Z,Z]$.

Fix $\epsilon ,\eta >0$. By [A2] choose $\delta >0$ such that the limiting upper probability of the displayed supremum exceeding $\epsilon$ is at most $\eta$. By [A1], $P^n(\mathrm{\parallel }{\widehat{\theta }}_n-{\theta }_0\mathrm{\parallel }\ge \delta )\to 0$. On the complementary event, $\mathrm{\parallel }{\mathbb{G}}_n({\psi }_{{\widehat{\theta }}_n}-{\psi }_{{\theta }_0})\mathrm{\parallel }$ is bounded by that supremum. Hence its limiting upper tail probability is at most $\eta$; letting $\eta \downarrow 0$ proves ${\mathbb{G}}_n({\psi }_{{\widehat{\theta }}_n}-{\psi }_{{\theta }_0})=o_P(1)$.

By [F12], applying the same measurable map ${\psi }_{{\theta }_0}$ to the iid observations preserves independence; identical distribution follows because every $X_i$ has the same law. By [A4] and [F5], ${\mathbb{G}}_n{\psi }_{{\theta }_0}=n^{-1\mathrm{/}2}{\sum }_{i=1}^n{\psi }_{{\theta }_0}(X_i)$ is a centered iid sum with finite covariance $\mathrm{\Sigma }$. AC is available by [F9], so [F7] gives ${\mathbb{G}}_n{\psi }_{{\theta }_0}\Rightarrow N_d(0,\mathrm{\Sigma })$, where convergence in distribution is weak convergence of laws by [F4]. In particular ${\mathbb{G}}_n{\psi }_{{\theta }_0}=O_P(1)$ by [F2].

Put $h_n={\widehat{\theta }}_n-{\theta }_0$ and $W_n=\sqrt{n}{h}_n$. By [F10]–[F11], the estimator and root have the stated Z-estimator and data-measurability conventions; [A1] imposes the stronger root-rate residual, which is also $o_P(1)$ since $n^{-1\mathrm{/}2}\le 1$. The identity $\sqrt{n}{P}_n{\psi }_{{\widehat{\theta }}_n}={\mathbb{G}}_n{\psi }_{{\widehat{\theta }}_n}+\sqrt{n}\mathrm{\Psi }({\widehat{\theta }}_n)$, [A1], step 1.1 and the derivative expansion in [A3], as specified by [F6], give $V{W}_n+\sqrt{n}\rho (h_n)=-{\mathbb{G}}_n{\psi }_{{\theta }_0}+o_P(1)$. Moreover $\mathrm{\parallel }\rho (h_n)\mathrm{\parallel }\mathrm{/}\mathrm{\parallel }{h}_n\mathrm{\parallel }=o_P(1)$ by [A1] and [A3], with the ratio set to zero when $h_n=0$.

Since $V$ is nonsingular, [A3] and step 2.1 imply $\mathrm{\parallel }{W}_n\mathrm{\parallel }\le \mathrm{\parallel }{V}^{-1}\mathrm{\parallel }(\mathrm{\parallel }{\mathbb{G}}_n{\psi }_{{\theta }_0}\mathrm{\parallel }+e_n+a_n\mathrm{\parallel }{W}_n\mathrm{\parallel })$, where $e_n=o_P(1)$ and $a_n=\mathrm{\parallel }\rho (h_n)\mathrm{\parallel }\mathrm{/}\mathrm{\parallel }{h}_n\mathrm{\parallel }=o_P(1)$ (set $a_n=0$ if $h_n=0$). With probability tending to one, $\mathrm{\parallel }{V}^{-1}\mathrm{\parallel }{a}_n\le 1\mathrm{/}2$. Absorbing that term and using ${\mathbb{G}}_n{\psi }_{{\theta }_0}=O_P(1)$ from step 1.2 proves $W_n=O_P(1)$.

Now $\sqrt{n}\rho (h_n)=o_P(1)$ because it is the product of $\mathrm{\parallel }{W}_n\mathrm{\parallel }=O_P(1)$ and $\mathrm{\parallel }\rho (h_n)\mathrm{\parallel }\mathrm{/}\mathrm{\parallel }{h}_n\mathrm{\parallel }=o_P(1)$. Substituting into step 2.1 gives $W_n=-V^{-1}{\mathbb{G}}_n{\psi }_{{\theta }_0}+o_P(1)$.

By [F4], [F7], and [F8], the continuous linear image $-V^{-1}{\mathbb{G}}_n{\psi }_{{\theta }_0}$ converges to a centered Gaussian with covariance $V^{-1}\mathrm{\Sigma }{V}^{-T}$: for every $u\in {\mathbb{R}}^d$, the projection of $N_d(0,\mathrm{\Sigma })$ along $-V^{-T}u$ is centered normal with variance $u^T{V}^{-1}\mathrm{\Sigma }{V}^{-T}u$. Applying [F3] to the $o_P(1)$ remainder in step 4.1 proves the asserted normal limit. AC is used in the multivariate iid CLT and the normal-law supplier, not in the stochastic-order or Taylor steps.