Finite-sample mean bias of the Nadaraya–Watson estimator

Write $h=h_n$, $Z_i=K((x-X_i)\mathrm{/}h)$, $B_i=Z_i(m(X_i)-m(x))$, $T=n^{-1}{\sum }_i{Z}_i$, $Q=n^{-1}{\sum }_i{B}_i$, $a=\mathbb{E}{Z}_1$, and $b=\mathbb{E}{B}_1$. In the compact/global alternatives, $m$ and ${\sigma }^2$ are bounded on the active kernel region for small $h$; thus $\mathbb{E}[\mathrm{\mid }{Y}_i\mathrm{\mid }\mid X_i]\le \mathrm{\mid }m(X_i)\mathrm{\mid }+\sqrt{{\sigma }^2(X_i)}$ is uniformly bounded there. The weighted response average has integrable absolute value, since it is bounded by ${\sum }_i\mathrm{\mid }{Y}_i\mathrm{\mid }{\mathbf{1}}_{\{Z_i>0\}}$. Conditioning on the predictors gives mean bias $\mathbb{E}[Q\mathrm{/}T;T>0]+(a_0-m(x))\mathbb{P}(T=0)$. Change of variables gives $a\asymp h$, and [F3] gives $c=b\mathrm{/}a=m_h(x)-m(x)=O(h^2)$. Bounded $m$ on the active region yields $\mathrm{\mid }{B}_i\mathrm{\mid }\le C{Z}_i$, $\mathbb{E}{Z}_i^2=O(h)$ and $\mathbb{E}(B_i-c{Z}_i{)}^2=O(h)$, by $R(K)<\mathrm{\infty }$ and the appropriate density bound. Under the rapid-tail alternative, [F6] gives $\int m^{2}f<\mathrm{\infty }$ and $\mathbb{E}\mathrm{\mid }Y\mathrm{\mid }<\mathrm{\infty }$, so integrability follows from the latter. The bounds $\mathbb{E}{Z}_i^2=O(h)$ and $\mathbb{E}(B_i-c{Z}_i{)}^2=O(h)$ still hold: inside a fixed smooth neighborhood $m,f$ are bounded; outside it $K((x-z)\mathrm{/}h{)}^2\le C{h}^6\mathrm{/}\mathrm{\mid }x-z{\mathrm{\mid }}^6$ and $\int m^{2}f\le \mathbb{E}{Y}^2<\mathrm{\infty }$. For $R=Q-cT$, independence gives $\mathbb{E}R=0$, $\mathbb{E}{R}^2=O(h\mathrm{/}n)$ and $\mathrm{Var}(T)=O(h\mathrm{/}n)$.

Positivity near zero implies there are $A,\kappa >0$ and a Borel set $J\subset [-A,A]$ of positive measure on which $K\ge \kappa$: the positive neighborhood is the countable union of its level sets $\{K\ge 1\mathrm{/}k\}$. Hence $p_h=\mathbb{P}((x-X_i)\mathrm{/}h\in J)\asymp h$. The count $N$ is binomial. On $G=\{N\ge n{p}_h\mathrm{/}2\}$, $T\ge c_{1}h$. With $\gamma =1-e^{-1}-1\mathrm{/}2>0$, Markov's bound applied to $e^{-N}$ gives $\mathbb{P}(G^c)\le e^{n{p}_h\mathrm{/}2}(1-p_h+p_h{e}^{-1}{)}^n\le e^{-\gamma n{p}_h}\le e^{-c_{2}nh}$. In particular $T=0$ is contained in $G^c$.

On $G$, $R\mathrm{/}T=R\mathrm{/}a-R(T-a)\mathrm{/}(aT)$. Since $\mathbb{E}R=0$, Cauchy–Schwarz bounds $\mathrm{\mid }\mathbb{E}[R\mathrm{/}a;G]\mathrm{\mid }$ by $a^{-1}(\mathbb{E}{R}^2{)}^{1\mathrm{/}2}\mathbb{P}(G^c{)}^{1\mathrm{/}2}$. The second term has absolute expectation at most $C{h}^{-2}(\mathbb{E}{R}^2{)}^{1\mathrm{/}2}\mathrm{Var}(T{)}^{1\mathrm{/}2}=O((nh{)}^{-1})$. On $G^c\cap \{T>0\}$, $\mathrm{\mid }R\mathrm{/}T\mathrm{\mid }\le C$ in the compact/global alternatives. In the rapid-tail alternative [F6] makes $m$ square-integrable, and the squared response-law mean is at most ${\sum }_{i}m(X_i{)}^2+a_0^2$. For each $i$, the event $G^c$ forces the count among the other $n-1$ observations below $n{p}_h\mathrm{/}2$; independence and the same binomial bound give $\mathbb{E}[m(X_i{)}^2;G^c]\le \mathbb{E}{Y}^2{e}^{-c_{3}nh}$. Thus Cauchy–Schwarz bounds its first-moment bad-event contribution by $O(\sqrt{n}{e}^{-c_{4}nh})$. The fallback and the omitted $c\mathbb{P}(T=0)$ also contribute $O(\mathbb{P}(G^c))$. Consequently $\mathbb{E}\widehat{m}-m(x)=c+O((nh{)}^{-1})+O((1+\sqrt{n}+(nh{)}^{-1\mathrm{/}2})e^{-c_{5}nh})$. Here $(nh{)}^{-1}=o(h^2)$ by $n{h}^3\to \mathrm{\infty }$, and $nh\ge h^{-2}$ and $nh\ge n^{2\mathrm{/}3}$ eventually make the exponential term $o(h^2)$. Substituting [F3] proves the claim.