Delete-one jackknife linearization for smooth functions of a mean

By [F1], ${\bar{X}}_n\to \mu$ and $n^{-1}{\sum }_i\mathrm{\parallel }{X}_i-{\bar{X}}_n{\mathrm{\parallel }}^2=\mathrm{tr}({\widehat{\mathrm{\Sigma }}}_n)$ is bounded in probability. For each $\epsilon >0$, [F2] and the finite second moment give $\mathbb{P}\text{\hspace{0.17em}⁣}({\max }_{i\le n}\mathrm{\parallel }{X}_i\mathrm{\parallel }>\epsilon n)\le n\mathbb{P}(\mathrm{\parallel }{X}_1\mathrm{\parallel }>\epsilon n)\le \frac{\mathbb{E}\mathrm{\parallel }{X}_1{\mathrm{\parallel }}^2}{{\epsilon }^{2}n}⟶0.$ Therefore ${\max }_i\mathrm{\parallel }{X}_i-{\bar{X}}_n\mathrm{\parallel }\mathrm{/}(n-1)\to 0$ in probability. Choose $\delta >0$ so the closed Euclidean ball $\bar{B}(\mu ,2\delta )$ is contained in $U$, and let $E_n=\{\mathrm{\parallel }{\bar{X}}_n-\mu \mathrm{\parallel }\le \delta ,{\max }_i\mathrm{\parallel }{X}_i-{\bar{X}}_n\mathrm{\parallel }\mathrm{/}(n-1)\le \delta \}$. This is measurable and has probability tending to one by the preceding bounds. Extend $\mathrm{\nabla }g$ by zero outside $U$ to a Borel map $\widetilde{\mathrm{\nabla }g}:{\mathbb{R}}^d\to {\mathbb{R}}^d$, and define ${\omega }_n={\max }_{i\le n}{\sup }_{t\in \mathbb{Q}\cap [0,1]}{1}_{E_n}\parallel \widetilde{\mathrm{\nabla }g}\text{\hspace{0.17em}⁣}({\bar{X}}_n-t\frac{X_i-{\bar{X}}_n}{n-1})-a\parallel \mathrm{.}$ Each rational-indexed term is measurable, and the countable supremum is measurable because its strict superlevel sets are countable unions; the finite maximum is measurable as well. On $E_n$, every segment lies in $\bar{B}(\mu ,2\delta )\subset U$, where $\mathrm{\nabla }g$ is bounded by continuity on this compact ball, and continuity along the segment makes the rational supremum equal the supremum over $[0,1]$. On $E_n^c$ every term inside the supremum is zero. The greatest distance of any segment point from $\mu$ on $E_n$ is at most $\mathrm{\parallel }{\bar{X}}_n-\mu \mathrm{\parallel }+{\max }_i\mathrm{\parallel }{X}_i-{\bar{X}}_n\mathrm{\parallel }\mathrm{/}(n-1)$, which tends to zero in probability. Continuity of $\mathrm{\nabla }g$ at $\mu$ and $\mathbb{P}(E_n)\to 1$ therefore imply ${\omega }_n\to 0$ in probability.

On $E_n$, define ${\phi }_i(t)=g({\bar{X}}_n-t(X_i-{\bar{X}}_n)\mathrm{/}(n-1))$ for $0\le t\le 1$. It is continuous on $[0,1]$ and differentiable on $(0,1)$ by [F5], with derivative $-(n-1{)}^{-1}\mathrm{\nabla }g({\bar{X}}_n-t(X_i-{\bar{X}}_n)\mathrm{/}(n-1){)}^T(X_i-{\bar{X}}_n)$. By [F4] there is ${\tau }_i\in (0,1)$ at which this derivative equals ${\phi }_i(1)-{\phi }_i(0)$. On every outcome define the measurable algebraic remainder $e_{n,i}:={\widehat{\theta }}_{n,(-i)}-{\widehat{\theta }}_n+a^T(X_i-{\bar{X}}_n)\mathrm{/}(n-1)$. On $E_n$ the mean-value identity expresses this remainder as $-\frac{(\mathrm{\nabla }g({\bar{X}}_n-{\tau }_i(X_i-{\bar{X}}_n)\mathrm{/}(n-1))-a{)}^T(X_i-{\bar{X}}_n)}{n-1}$, so $\mathrm{\mid }{e}_{n,i}\mathrm{\mid }\le {\omega }_n\mathrm{\parallel }{X}_i-{\bar{X}}_n\mathrm{\parallel }\mathrm{/}(n-1)$. The points ${\tau }_i$ are used only for this pointwise identity and bound; no measurable selection is needed. Since $\mathbb{P}(E_n)\to 1$, the expansion with the stated bound holds with probability tending to one.

The algebraic definition of $e_{n,i}$ gives $p_{n,i}={\widehat{\theta }}_n+a^T(X_i-{\bar{X}}_n)-(n-1)e_{n,i}$. On $E_n$, the bound in step 2.1 makes the average squared error in the statement at most ${\omega }_n^2\frac{1}{n}{\sum }_i\mathrm{\parallel }{X}_i-{\bar{X}}_n{\mathrm{\parallel }}^2,$ which tends to zero in probability by step 1.1. Since $\mathbb{P}(E_n^c)\to 0$, the stated convergence in probability follows.

The algebraic definition of $e_{n,i}$ gives, after averaging over $i$ and using ${\sum }_i(X_i-{\bar{X}}_n)=0$, $n{V}_{J,n}=\frac{1}{n-1}{\sum }_i{(a^T(X_i-{\bar{X}}_n)-(n-1)(e_{n,i}-{\bar{e}}_n))}^2,$ where ${\bar{e}}_n=n^{-1}{\sum }_i{e}_{n,i}$. The leading quadratic term is $\frac{1}{n-1}{\sum }_i\{a^T(X_i-{\bar{X}}_n){\}}^2=\frac{n}{n-1}{a}^T{\widehat{\mathrm{\Sigma }}}_{n}a⟶a^T\mathrm{\Sigma }a$ by [F1]. On $E_n$, the normalized squared error term is bounded by $(n-1){\sum }_i(e_{n,i}-{\bar{e}}_n{)}^2\le (n-1){\sum }_i{e}_{n,i}^2\le {\omega }_n^2\frac{1}{n-1}{\sum }_i\mathrm{\parallel }{X}_i-{\bar{X}}_n{\mathrm{\parallel }}^2⟶0.$ If ${\psi }_i=a^T(X_i-{\bar{X}}_n)$ and $r_i=(n-1)(e_{n,i}-{\bar{e}}_n)$, then on $E_n$ the absolute cross term is at most $2\sqrt{\left(\frac{1}{n-1}{\sum }_i{\psi }_i^2\right)\left(\frac{1}{n-1}{\sum }_i{r}_i^2\right)},$ which tends to zero because the first factor converges to the finite $a^T\mathrm{\Sigma }a$ and the second to zero. Since $\mathbb{P}(E_n^c)\to 0$, these bounds prove the jackknife variance limit in probability.