Essential boundedness does not imply pointwise uniform universality

The spike set $\{1\mathrm{/}n:n\in \mathbb{N},n\ge 1\}$ is countable and null by [F3]. Thus $\sigma =H$ almost everywhere and $\sigma$ is locally essentially bounded. Its discontinuities are exactly $\{0\}\cup \{1\mathrm{/}n:n\in \mathbb{N},n\ge 1\}$, a closed null set: away from this set the step part is locally constant and there are no spikes nearby. Hence the activation satisfies [F4].

Consider a finite network $N(x)=c+{\sum }_j{a}_j\sigma (w_{j}x+b_j)$ on $[-1,1]$. Absorb constant affine units into $c$, combine terms with identical affine maps and discard groups with zero combined coefficient. Call a nonconstant affine map active when its zero lies in $[-1,1]$ and it takes positive values arbitrarily close to that zero at points of $[-1,1]$. If an active group remains, fix its zero $x_0$ and choose, among active groups at $x_0$ with nonzero coefficient, one with minimal $\mathrm{\mid }{w}_0\mathrm{\mid }$; write its combined coefficient as $A_0\mathrm{\ne }0$.

The function $\sigma$ is not polynomial almost everywhere. If a polynomial agreed with it almost everywhere, it would vanish almost everywhere on $(-\mathrm{\infty },0)$, so [F2] would force it to be the zero polynomial; this contradicts $\sigma =1$ almost everywhere on $(0,1)$.

Set $x_n=x_0+1\mathrm{/}(n{w}_0)$, choosing $n$ sufficiently large that $x_n\in [-1,1]$. The chosen unit has preactivation $1\mathrm{/}n$ and contributes $A_0(n^2+1)$. For a unit with a different zero, its preactivation tends to a nonzero value; if the limit is negative the unit is eventually negative, and if positive it lies near only finitely many reciprocals, each attained at most once by the nonconstant affine sequence. Thus it spikes at only finitely many $x_n$. At the same zero, an active group with opposite sign has negative preactivation on this sequence; one with the same sign and a distinct affine map has slope ratio $r=\mathrm{\mid }w\mathrm{\mid }\mathrm{/}\mathrm{\mid }{w}_0\mathrm{\mid }>1$. If $r$ is irrational the equation $r\mathrm{/}n=1\mathrm{/}m$ is impossible; if $r=p\mathrm{/}q>1$ in lowest terms it requires $p\mid n$. Choose the infinite sequence $n=1+kP$, where $P$ is the product of the finitely many such numerators $p$ (take $P=1$ if there are none); then none of those other groups spikes. Along a tail of this sequence all other spike terms vanish and all Heaviside contributions remain bounded, so $\mathrm{\mid }N(x_n)\mathrm{\mid }\to \mathrm{\infty }$. Thus a bounded network can have no active group with nonzero coefficient.

For a bounded network, every nonconstant group that remains has no positive threshold crossing in $[-1,1]$. Its Heaviside part is constant on $(-1,1)$, and its spike set in $[-1,1]$ is finite because its positive preactivation values stay bounded away from zero. Constant units are constant as well. Hence $N$ is constant outside a finite subset of $[-1,1]$.

If $N$ is unbounded, it cannot approximate the bounded target $f(x)=x$ uniformly. If it is bounded, choose $x_{-},x_{+}$ outside the finite exceptional set with $x_{+}-x_{-}>1$; then $N(x_{-})=N(x_{+})$, so at least one of $\mathrm{\mid }N(x_{-})-x_{-}\mathrm{\mid }$ and $\mathrm{\mid }N(x_{+})-x_{+}\mathrm{\mid }$ exceeds $1\mathrm{/}2$. Thus no network approximates $x$ on $[-1,1]$ with error below $1\mathrm{/}2$, refuting the pointwise-uniform claim.