Conditional asymptotic variance of Nadaraya–Watson and local-linear estimators

By [F3], $M_n=(nh{)}^{-1}{\sum }_{i}K(U_i)(1,U_i{)}^{\mathsf{T}}(1,U_i)\to f(x)\mathrm{diag}(1,{\mu }_2)$ in probability. Put $V_n=(nh{)}^{-1}{\sum }_{i}K(U_i{)}^2{\sigma }^2(X_i)(1,U_i{)}^{\mathsf{T}}(1,U_i)$. The scalar generalization proved in [F3] applies also to a continuous multiplier ${\sigma }^2(x-hu)$: on fixed truncated support this multiplier converges uniformly to ${\sigma }^2(x)$ and is bounded, and the tail expectation is bounded by $C{\int }_{\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{l}}\mathrm{\mid }L\mathrm{\mid }$. Thus, for $L=K^2$ the $(0,0)$ entry tends to $f(x){\sigma }^2(x)R(K)$. In the rapid-tail alternative, local truncation remains valid. For $j=0,1,2$ the expected absolute far contribution to $V_{n,j}$ is bounded by $C{h}^{5-j}{\int }_{\mathrm{\mid }z-x\mathrm{\mid }>\delta }\mathrm{\mid }z-x{\mathrm{\mid }}^{j-6}{\sigma }^2(z)f(z)dz=o(1)$, since [F7] gives $\int {\sigma }^{2}f\le \mathbb{E}{Y}^2$. Also $\int (1+u^2)K^2<\mathrm{\infty }$ follows from boundedness and the rapid tail. For the local-linear conclusion, $\int (1+u^2)K^2<\mathrm{\infty }$ permits $L=K^2{u}^j$, $j=0,1,2$, by the same truncation argument. All entries then converge to $f(x){\sigma }^2(x)\int K^2{u}^{j+k}$; symmetry makes the off-diagonal limit zero. Terms with $K(U_i)=0$ are interpreted as zero. Every active response-law variance is finite almost surely; these are variances under the specified conditional response laws, without an assumption of unconditional square integrability.

On the event ${\sum }_{i}K(U_i)>0$, [F1], conditional response independence, and [F4] give $\mathrm{Var}({\widehat{m}}_{\mathrm{N}\mathrm{W},h_n}(x)\mid X_{1:n})={\displaystyle \frac{\sum _{i}K(U_i{)}^2{\sigma }^2(X_i)}{(\sum _{i}K(U_i){)}^2}}$. Writing $M_{n,00}=(n{h}_n{)}^{-1}{\sum }_{i}K(U_i)$ and $V_{n,00}=(n{h}_n{)}^{-1}{\sum }_{i}K(U_i{)}^2{\sigma }^2(X_i)$ makes the scaled variance $V_{n,00}\mathrm{/}{M}_{n,00}^2$, which converges to $f(x){\sigma }^2(x)R(K)\mathrm{/}f(x{)}^2$. The zero-denominator event has probability tending to zero by [F3]'s $(0,0)$ entry, so the same convergence holds with the fallback.

On a nonsingular local-linear design matrix, weighted least squares gives conditional variance $(n{h}_n{)}^{-1}{e}_0^{\mathsf{T}}{M}_n^{-1}{V}_n{M}_n^{-1}{e}_0$, where $e_0=(1,0{)}^{\mathsf{T}}$. The limit matrix is diagonal with entries $f(x)$ and $f(x){\mu }_2$, so its inverse has first diagonal entry $1\mathrm{/}f(x)$; the $(0,0)$ limit of $V_n$ is $f(x){\sigma }^2(x)R(K)$. Matrix inversion is continuous at this invertible limit, because its entries are quotients of entries of the adjugate by the nonzero determinant. Thus the scaled conditional variance converges to $R(K){\sigma }^2(x)\mathrm{/}f(x)$. By [F3] the singular-design event has probability tending to zero, and the fallback does not change convergence in probability.

For the extra Nadaraya–Watson clause, choose $A,\kappa >0$ with $J=\{\mathrm{\mid }u\mathrm{\mid }\le A,K(u)\ge \kappa \}$ of positive measure, possible since $\int K=1$. Its predictor count is binomial with success probability $p_h\asymp h$. On $G_n=\{N\ge n{p}_h\mathrm{/}2\}$, $T_n=n^{-1}{\sum }_{i}K(U_i)\ge c_{1}h$, whereas Markov's bound applied to $e^{-N}$ gives $\mathbb{P}(G_n^c)\le \exp (-c_{2}nh)$. The truncation argument of step 1.1 also proves $V_{n,00}\to f(x){\sigma }^2(x)R(K)$ in $L^1$: the bounded truncated average converges in $L^2$, while the expected absolute tail is uniformly small. On $G_n$, $(T_n\mathrm{/}h{)}^{-2}$ is bounded and converges in probability to $f(x{)}^{-2}$. It follows that the expectation of the scaled conditional variance on $G_n$ has the stated limit: separate $V_{n,00}$ into its constant limit and an $L^1$ error, and use bounded convergence in probability for the reciprocal factor. On $G_n^c$, the conditional variance is at most the active-region bound on ${\sigma }^2$, since the weights are nonnegative and sum to one, or the fallback is deterministic. Hence its scaled expectation is $O(nh{e}^{-c_{2}nh})\to 0$ in those alternatives. Under rapid tails, the conditional variance is at most ${\sum }_i{\sigma }^2(X_i)$; the count among the other $n-1$ observations is independent of $X_i$ and is below $n{p}_h\mathrm{/}2$ on $G_n^c$. Its binomial lower-tail bound gives $\mathbb{E}[{\sigma }^2(X_i);G_n^c]\le \mathbb{E}{Y}^2{e}^{-c_{3}nh}$. Thus the contribution to the expected scaled conditional variance is at most $n^{2}h\mathbb{E}{Y}^2{e}^{-c_{3}nh}$. Under the stated $n{h}^3\to \mathrm{\infty }$ condition, $nh\ge n^{2\mathrm{/}3}$ eventually, so this polynomial factor times the exponential tends to zero.

Let $A_n$ be the response-law mean of the Nadaraya–Watson fit, including its fallback, and let $m_h=\int Kmf\mathrm{/}\int Kf$. The kernel integrals and localization or global bounds imply $m_h\to m(x)$. Put $Z_i=K(U_i)$ and $R_n=n^{-1}{\sum }_i{Z}_i(m(X_i)-m_h)$. Then $\mathbb{E}{R}_n=0$ and $\mathbb{E}{R}_n^2\le n^{-1}\mathbb{E}[Z_1^2(m(X_1)-m_h{)}^2]=o(h\mathrm{/}n)$: after change of variables, continuity at $x$ handles a fixed compact $u$-set and the integrable $K^2$ tail is dominated by bounded $f,m$. On $G_n$, $A_n-m_h=R_n\mathrm{/}{T}_n$, so its expected square is $o((nh{)}^{-1})$. On $G_n^c$, $A_n$ and $m_h$ are bounded in the compact/global alternatives. Under rapid tails, [F7] gives $\int m^{2}f<\mathrm{\infty }$; the far part of the second-moment integral is $O(h^6)\int m^{2}f=o(h)$, and the leave-one-out binomial argument in step 3.1 gives $\mathbb{E}[A_n^2;G_n^c]\le n\mathbb{E}{Y}^2{e}^{-c_{4}nh}+a_0^2\mathbb{P}(G_n^c)$. Its expected squared bad-event contribution is therefore negligible in all stated cases, and $\mathrm{Var}(A_n)\le \mathbb{E}(A_n-m_h{)}^2=o((nh{)}^{-1})$. Bounded active-region means and variances, or the bound ${\widehat{m}}^2\le {\sum }_i{Y}_i^2+a_0^2$ under rapid tails, make the Nadaraya–Watson fit square-integrable. Expanding its centered square and integrating first under the response laws gives total variance equal to the expected conditional variance plus $\mathrm{Var}(A_n)$. Step 3.1 therefore proves the unconditional Nadaraya–Watson expansion.