A one-neuron kernel has convergent uniform quadratures and divergent C1 quadratures

For integers $n\ge 2$ put $t_n=3\cdot 2^{-n-2}$, $w_n=2^{-4n-4}$, $H_n=2^{2n}$ and ${\rho }_n(s)=H_n\beta ((s-t_n)\mathrm{/}{w}_n)$. Their supports are disjoint subintervals of $(0,1)$: successive centers differ by $3\cdot 2^{-n-3}$, which exceeds $w_n+w_{n+1}$, and each radius is smaller than its center. Define $\rho (s)={\sum }_{n\ge 2}{\rho }_n(s)$ for $s>0$, and $\rho (s)=0$ for $s\le 0$. On each compact subinterval of $(0,\mathrm{\infty })$ only finitely many summands are nonzero. Moreover $m_n=\int {\rho }_n\le 2H_n{w}_n=2^{-2n-3}$, so $B={\sum }_{n\ge 2}{m}_n<1$. Define $\sigma (s)={\int }_0^s\rho (t)dt$ for $s\ge 0$ and $\sigma (s)=0$ for $s<0$. The tail mass bound proves continuity at zero and uniform convergence of the primitives ${\sigma }_n(s)={\int }_0^s{\rho }_n(t)dt$; thus $\sigma =\sum {\sigma }_n$ is continuous on $\mathbb{R}$, bounded by $B$, and smooth on $(0,\mathrm{\infty })$. It is a legitimate continuous activation. It is bounded and nonconstant, hence is not polynomial even almost everywhere: continuity would make an almost-everywhere polynomial identity hold everywhere, and a bounded polynomial is constant. Both $\sigma$ and ReLU are continuous, hence Borel measurable with empty discontinuity set, and pointwise bounded on every compact interval (globally for $\sigma$, and by the interval radius for ReLU). They therefore belong to ${\mathcal{M}}_{\mathrm{p}\mathrm{t}}$ in [F3]. ReLU is not polynomial almost everywhere: continuity would give a polynomial identity everywhere, while vanishing on the negative half-line forces that polynomial to be zero, contradicting its positive values. Applying [F3] in each input dimension now places $\sigma$ in every ${\mathcal{A}}_0$ class and ReLU in every ${\mathcal{A}}_0^L$ class; ReLU is Borel and obeys $\mathrm{\mid }\mathrm{ReLU}(s)\mathrm{\mid }\le \mathrm{\mid }s\mathrm{\mid }$.

Set $\kappa (x,y)=\sigma (x+y)$ and $G(a)(x)={\int }_0^1\sigma (x+y)a(y)dy$. The kernel is exactly a one-hidden-unit network on $(x,y)$ with weights $(1,1)$, activation $\sigma$, output weight $1$, and zero biases. The other activations can be the identity. Each $G_n^{\ast }(a)(x)={\int }_0^1{\sigma }_n(x+y)a(y)dy$ is $C^1$, with derivative ${\int }_0^1{\rho }_n(x+y)a(y)dy$, and both its function norm and derivative norm are at most $m_n\mathrm{\parallel }a{\mathrm{\parallel }}_{\mathrm{\infty }}$. Consequently both the function series and derivative series converge uniformly. To verify differentiation without an interchange assumption, the finite partial sums satisfy $S_N(x)-S_N(0)={\int }_0^x{S}_N^{\mathrm{\prime }}(t)dt$; uniform convergence permits passage to the limit in this equality, and the fundamental theorem of calculus gives a continuous derivative. Hence $G:A\to U$ is bounded linear, with $\mathrm{\parallel }G(a){\mathrm{\parallel }}_U\le 2B\mathrm{\parallel }a{\mathrm{\parallel }}_{\mathrm{\infty }}$. Both $A$ and $U$ embed continuously into $C([0,1])$. To instantiate the source integral subclass as well, take $S_0$ and $S_1$ to have constant kernel $1$ and zero bias, and use $\kappa (x,y)=\sigma (x+y)$ as the final kernel, choosing ReLU for the input and intermediate kernel networks and activations. No intermediate activation occurs at depth $N=2$. This yields $\widehat{G}(a)=({\int }_0^{1}a)G(1)$, a bounded map $A\to U$. For $a=1$, both preceding integral layers and their quadratures are exactly the constant $1$, so its final continuum and quadrature outputs are precisely those analyzed below. Constant kernels are exact networks through their affine output biases.

At refinement $n$ partition $(0,1)$ into $2^n$ dyadic cells of length $h_n=2^{-n}$. Choose the first tag to be $t_n\in (0,h_n)$ and choose the midpoint of every other cell. Let $Q_n(a)(x)=h_n{\sum }_{j=1}^{2^n}\sigma (x+{\xi }_{n,j})a({\xi }_{n,j})$. These are compatible tagged Riemann sums, with mesh diameter tending to zero and the same continuum parameters at every refinement. Since every tag is positive, each summand is smooth in a neighborhood of $[0,1]$, so $Q_n(a)\in U$. For $a=1$, all derivative summands are nonnegative and $Q_n(1{)}^{\mathrm{\prime }}(0)\ge h_n\rho (t_n)=h_n{H}_n=2^n$. On the other hand, substitution in the continuum integral gives $G(1)(x)={\int }_x^{x+1}\sigma (s)ds$ and $G(1{)}^{\mathrm{\prime }}(0)=\sigma (1)-\sigma (0)=B$. Thus $\mathrm{\parallel }{Q}_n(1)-G(1){\mathrm{\parallel }}_U\ge 2^n-B\to \mathrm{\infty }$.

The set $K=\{1\}\subset A$ is compact. The continuous integrand $(x,y)\mapsto \sigma (x+y)$ on $[0,1{]}^2$ is uniformly continuous: otherwise pairs at distances tending to zero with a fixed positive function gap have convergent subsequences in the compact square and contradict continuity. Writing its modulus of continuity as $\omega$, the cellwise estimate gives $\mathrm{\parallel }{Q}_n(1)-G(1){\mathrm{\parallel }}_{\mathrm{\infty }}\le \omega (h_n)\to 0$. Thus the genuine supremum-norm conclusion coexists with divergence in the admissible output norm, refuting the asserted inference.