Pointwise second-order bias of the KDE

By [F2], $Z_i:=K((x-X_i)\mathrm{/}h)$ is a measurable real random variable. By [F3], $f$ is bounded. Applying [F5] to $\mathrm{\mid }{Z}_i\mathrm{\mid }$, then [F6] to $u\mapsto x-hu$, gives $\mathbb{E}\mathrm{\mid }{Z}_i\mathrm{\mid }=\int \mathrm{\mid }K((x-t)\mathrm{/}h)\mathrm{\mid }f(t)dt=h^d\int \mathrm{\mid }K(u)\mathrm{\mid }f(x-hu)du\le h^d\mathrm{\parallel }f{\mathrm{\parallel }}_{\mathrm{\infty }}\mathrm{\parallel }K{\mathrm{\parallel }}_1<\mathrm{\infty }\mathrm{.}$ Thus every $Z_i$ is integrable. Put $g(t)=K((x-t)\mathrm{/}h)$ and $g^{+}=\max (g,0)$, $g^{-}=\max (-g,0)$. Both parts are nonnegative and their density integrals are finite, being bounded by the absolute integral just established. Apply [F5] separately to $g^{+}$ and $g^{-}$, then subtract their finite integrals using [F7]: $\mathbb{E}{Z}_i=\int g^{+}f-\int g^{-}f=\int gf$. This derives the signed density identity without applying the nonnegative density theorem directly to $g$. Since the $X_i$ have the same law, the $Z_i$ have the same expectation. Linearity [F7] and the signed substitution in [F6] therefore yield $\mathbb{E}{\widehat{f}}_h(x)=\frac{1}{h^d}\mathbb{E}{Z}_1={\int }_{{\mathbb{R}}^d}K(u)f(x-hu)du\mathrm{.}$

For fixed $u$, set $q(t)=f(x-thu)$ on $[0,1]$. By [F9], $q^{\mathrm{\prime }}(0)=-h\mathrm{\nabla }f(x)\cdot u$ and $q^{\mathrm{\prime }\mathrm{\prime }}(t)=h^2{u}^T{D}^{2}f(x-thu)u$. Integration by parts with $1-t$ and $q^{\mathrm{\prime }}$, then the second fundamental theorem, gives $f(x-hu)=f(x)-h\mathrm{\nabla }f(x)\cdot u+h^2{\int }_0^1(1-t)u^T{D}^{2}f(x-thu)udt\mathrm{.}$ For $u\mathrm{\ne }0$, subtracting $\frac{h^2}{2}{u}^T{D}^{2}f(x)u$ defines a remainder $h^2\mathrm{\parallel }u{\mathrm{\parallel }}^2\rho (h,u)$, where $\rho (h,u)={\int }_0^1(1-t)\frac{u^T(D^{2}f(x-thu)-D^{2}f(x))u}{\mathrm{\parallel }u{\mathrm{\parallel }}^2}dt\mathrm{.}$ Continuity of the Hessian at $x$ gives $\rho (h,u)\to 0$ for each fixed $u$; the global Hessian bound gives $\mathrm{\mid }\rho (h,u)\mathrm{\mid }\le H$ uniformly in $h,u$. Set $\rho (h,0)=0$.

Substitute the expansion from step 1.2 into the convolution in step 1.1. The first moments are absolutely integrable because $\mathrm{\mid }{u}_j\mathrm{\mid }\le 1+\mathrm{\parallel }u{\mathrm{\parallel }}^2$ and $K\in L^1$ with finite second absolute moment, so unit mass cancels $f(x)$ and the zero first moments cancel the linear term. The remaining quadratic term is $\frac{h^2}{2}{\sum }_{j,k}{\mu }_{jk}{\mathrm{\partial }}_{jk}f(x)$. Moreover, $\mathrm{\mid }K(u)\mathrm{\mid }\mathrm{\parallel }u{\mathrm{\parallel }}^{2}H$ is integrable and dominates the scaled remainder. Applying [F8] as $h\downarrow 0$ shows that the remainder divided by $h^2$ tends to zero. The second moments ${\mu }_{jk}$ are absolutely finite by the same integrable moment condition.