Neural operators approximate continuous maps uniformly on compacta

If $K$ is empty choose the zero operator. Otherwise put $T=G^{†}{\mathrm{\mid }}_K$ and choose $0<\tau <\epsilon \mathrm{/}5$. Every listed input space injects continuously into $L^1(D)$: for finite $p$, $\mathrm{\parallel }a{\mathrm{\parallel }}_1\le \mathrm{\mid }D{\mathrm{\mid }}^{1-1\mathrm{/}p}\mathrm{\parallel }a{\mathrm{\parallel }}_p\le \mathrm{\mid }D{\mathrm{\mid }}^{1-1\mathrm{/}p}\mathrm{\parallel }a{\mathrm{\parallel }}_{W^{m,p}}$ when applicable, and for $C(\bar{D})$ use $\mathrm{\parallel }a{\mathrm{\parallel }}_1\le \mathrm{\mid }D\mathrm{\mid }\mathrm{\parallel }a{\mathrm{\parallel }}_{\mathrm{\infty }}$. Thus each smooth moment $a\mapsto {\int }_D\phi a$ is continuous on $A$ and such moments separate distinct members of $K$, using [F4]; distinct continuous functions also differ on an interior set of positive measure.

ReLU is continuous and Borel, is pointwise bounded on every compact interval, and has empty discontinuity set, so it belongs to ${\mathcal{M}}_{\mathrm{p}\mathrm{t}}$. It cannot agree almost everywhere with a polynomial: that polynomial would vanish on the negative half-line by continuity, hence identically, contradicting ReLU on the positive half-line. Thus [F6] applies. The corrected property has concrete instances. For ReLU and a compact $S\subset {\mathbb{R}}^q$, suppose $C\ge {\sup }_S\mathrm{\mid }z{\mathrm{\mid }}_2$. If $S$ is empty choose zero. If $C=0$, then $S\subset \{0\}$ and again choose zero. If $C>0$, the coordinate clamp $c_j(z)=-C+\mathrm{ReLU}(z_j+C)-\mathrm{ReLU}(z_j-C)$ is an exact ReLU network onto $[-C,C{]}^q$ and fixes $S$. The radial projection $P$ onto the Euclidean ball of radius $C$ is continuous and fixes $S$. By [F6] approximate $P$ on the cube by a ReLU MLP $p$ with Euclidean error $<\xi$, where $0<2\xi <\eta$. Then $b(z)=Cp(c(z))\mathrm{/}(C+\xi )$ is a finite ReLU MLP, globally bounded by $C$, and for $z\in S$ its error is less than $\xi +\xi C\mathrm{/}(C+\xi )<2\xi <\eta$. Composition of layered MLPs is a layered MLP because consecutive affine maps merge. ReLU also has linear growth and belongs to every relevant ${\mathcal{A}}_0$ by [F6]. Thus this corrected branch is nonvacuous.

The set $Q=\{(a,b)\in K^2:\mathrm{\parallel }T(a)-T(b){\mathrm{\parallel }}_U\ge \tau \}$ is compact. For each pair in $Q$ a separating moment from step 1.1 remains nonzero on a neighborhood of that pair. A finite subcover supplies moments ${\phi }_1,\dots ,{\phi }_J$ such that $E(a)=({\int }_D{\phi }_{j}a{)}_{j=1}^J$ has $E(a)\mathrm{\ne }E(b)$ for all $(a,b)\in Q$. If $Q$ is nonempty, the positive continuous function $\mathrm{\mid }E(a)-E(b){\mathrm{\mid }}_2$ on $Q$ has a positive minimum $r$; if $Q$ is empty set $J=1$, ${\phi }_1=0$, and $r=1$. In both cases $\mathrm{\mid }E(a)-E(b){\mathrm{\mid }}_2<r$ implies $\mathrm{\parallel }T(a)-T(b){\mathrm{\parallel }}_U<\tau$. This is the complete finite-moment reduction and does not presume a dual representation theorem.

Cover the compact $E(K)$ by finitely many open balls of radius $r\mathrm{/}4$ with centers $z_i=E(a_i)$, $a_i\in K$, $1\le i\le s$. Put $h_i(z)=\max (0,r\mathrm{/}2-\mathrm{\mid }z-z_i{\mathrm{\mid }}_2)$, $h_0(z)=\mathrm{dist}(z,E(K))$, and $w_i(z)=h_i(z)\mathrm{/}(h_0(z)+{\sum }_{k=1}^s{h}_k(z))$. The denominator is positive everywhere: off $E(K)$ its first term is positive, and on $E(K)$ some $h_i>r\mathrm{/}4$. Consequently $w=(w_1,\dots ,w_s)$ is continuous on all ${\mathbb{R}}^J$, nonnegative, and ${\sum }_i{w}_i\le 1$, with equality on $E(K)$. For $a\in K$, $w_i(E(a))>0$ implies $\mathrm{\mid }E(a)-E(a_i){\mathrm{\mid }}_2<r\mathrm{/}2$ and hence $\mathrm{\parallel }T(a)-T(a_i){\mathrm{\parallel }}_U<\tau$. The convex combination ${\sum }_i{w}_i(E(a))T(a_i)$ therefore differs from $T(a)$ by less than $\tau$.

For every listed output space choose smooth neighborhood restrictions ${\varphi }_i$ with $\mathrm{\parallel }{\varphi }_i-T(a_i){\mathrm{\parallel }}_U<\tau$, using [F3]. Define $Bv={\sum }_i{v}_i{\varphi }_i$ and $L=\max (1,\mathrm{\parallel }B{\mathrm{\parallel }}_{{\mathrm{\ell }}^1\to U})$. Since the weights in step 3.1 sum to one on $E(K)$, ${\sup }_K\mathrm{\parallel }T(a)-Bw(E(a)){\mathrm{\parallel }}_U<2\tau$. This is the output finite-dimensional factorization.

Let $Z=\{z:\mathrm{dist}(z,E(K))\le 1\}$, a compact subset of ${\mathbb{R}}^J$. Uniform continuity of $w$ on $Z$ gives $0<\delta <1$ such that $z,z^{\mathrm{\prime }}\in Z$ and $\mathrm{\mid }z-z^{\mathrm{\prime }}{\mathrm{\mid }}_2<\delta$ imply $\mathrm{\mid }w(z)-w(z^{\mathrm{\prime }}){\mathrm{\mid }}_1<\tau \mathrm{/}L$. Set $R=\max (1,{\sup }_{a\in K}\mathrm{\parallel }a{\mathrm{\parallel }}_1)$. By the ${\sigma }_2$ premise [F1] choose common-depth scalar networks $p_j$ with ${\sup }_{\bar{D}}\mathrm{\mid }{p}_j-{\phi }_j\mathrm{\mid }<\delta \mathrm{/}(2R\sqrt{J})$. Then $\tilde{E}(a)=({\int }_D{p}_j(y)a(y)dy{)}_j$ satisfies ${\sup }_K\mathrm{\mid }\tilde{E}-E{\mathrm{\mid }}_2<\delta \mathrm{/}2$, so $\tilde{E}(K)\subset Z$ and ${\sup }_K\mathrm{\parallel }Bw(E(a))-Bw(\tilde{E}(a)){\mathrm{\parallel }}_U<\tau$. The input layer kernel $(x,y)\mapsto (p_j(y){)}_j$ ignores $x$ and has exactly constant spatial output. The estimate uses only $L^1$ bounds and covers $p_1=1$.

Apply the ${\sigma }_1\in {\mathcal{A}}_0$ premise to the continuous map $w$ on compact $\tilde{E}(K)$, obtaining a finite MLP $f:{\mathbb{R}}^J\to {\mathbb{R}}^s$ such that ${\sup }_{z\in \tilde{E}(K)}\mathrm{\mid }f(z)-w(z){\mathrm{\mid }}_1<\tau \mathrm{/}L$. Then ${\sup }_K\mathrm{\parallel }Bw(\tilde{E}(a))-Bf(\tilde{E}(a)){\mathrm{\parallel }}_U<\tau$. Only $C^0$ approximation of this middle map is used; no differentiability of ${\sigma }_1$ is needed.

Put $V=\max (1,{\sup }_{a\in K}\mathrm{\mid }f(\tilde{E}(a)){\mathrm{\mid }}_1)$, finite by continuity and compactness. Define $c_U=1$ for $U=C^{m_2}({\bar{D}}^{\mathrm{\prime }})$ with the maximum derivative norm, and $c_U=(J_{m_2}\mathrm{\mid }{D}^{\mathrm{\prime }}\mathrm{\mid }{)}^{1\mathrm{/}{p}_2}$ for $U=W^{m_2,p_2}(D^{\mathrm{\prime }})$ with the usual sum-of-$p_2$-powers norm, where $J_m$ counts multi-indices of order at most $m$; this also treats $L^{p_2}$ with $J_0=1$. For $C^{m_2}$ functions $u$, directly integrating each derivative gives $\mathrm{\parallel }u{\mathrm{\parallel }}_U\le c_U\mathrm{\parallel }u{\mathrm{\parallel }}_{C^{m_2}}$. By ${\sigma }_3\in {\mathcal{A}}_{m_2}$ choose common-depth networks $q_i$ such that $\mathrm{\parallel }{q}_i-{\varphi }_i{\mathrm{\parallel }}_{C^{m_2}({\bar{D}}^{\mathrm{\prime }})}<\tau \mathrm{/}(c_{U}V)$. These approximants are chosen from the $C^{m_2}$ network subfamily supplied by that density premise, so their derivatives exist. With $\tilde{B}v={\sum }_i{v}_i{q}_i$, the estimate $\mathrm{\parallel }(\tilde{B}-B)v{\mathrm{\parallel }}_U\le c_U\mathrm{\mid }v{\mathrm{\mid }}_1{\max }_i\mathrm{\parallel }{q}_i-{\varphi }_i{\mathrm{\parallel }}_{C^{m_2}}$ yields ${\sup }_K\mathrm{\parallel }\tilde{B}f(\tilde{E}(a))-Bf(\tilde{E}(a)){\mathrm{\parallel }}_U<\tau$. Equivalent choices of standard Banach norms merely change $c_U$.

Write the middle MLP as $f(z)=A_{n+1}{\sigma }_1(A_n{\sigma }_1(\cdots {\sigma }_1(A_{1}z+b_1)\cdots )+b_n)+b_{n+1}$ with $n$ hidden layers. On a spatially constant field $v(y)=z$, the affine map $z\mapsto Az+b$ is represented exactly by the integral kernel $A\mathrm{/}\mathrm{\mid }D\mathrm{\mid }$ and constant bias $b$. Such constants are exact ${\sigma }_2$ networks through their output biases. Use these layers after the input layer of step 5.1, placing precisely the $n$ coordinatewise ${\sigma }_1$ activations of $f$ between them. The final kernel $\kappa (x,y)=\mathrm{\mid }D{\mathrm{\mid }}^{-1}(q_1(x),\dots ,q_s(x))$ is a ${\sigma }_3$ network ignoring $y$, so its integral over $D$ sends a constant $v$ to $\tilde{B}v$. This proves that $G(a)=\tilde{B}f(\tilde{E}(a))$ belongs to ${\mathrm{N}\mathrm{O}}_{n+2}$ under the indexing convention in [F2]. All intermediate fields are constant and the final output belongs to $U$ for every $a\in A$.

Adding the four errors in steps 4.1–7.1 gives ${\sup }_K\mathrm{\parallel }T(a)-G(a){\mathrm{\parallel }}_U<5\tau <\epsilon$. This proves the core theorem. The proof in fact only needs continuity of $T$ on $K$ and the continuous injective $L^1$ realization of $A$; it retains all the listed source spaces.

For the corrected additional branch assume $U$ Hilbert, $M>0$, and ${\sup }_A\mathrm{\parallel }{G}^{†}(a){\mathrm{\parallel }}_U\le M$. Apply the core construction with an error $0<\gamma <\min (M,\epsilon \mathrm{/}2)$, calling the resulting operator $G_0$. It has the exact form $G_0(a)={\sum }_i{f}_i(\tilde{E}(a))q_i$. Choose an orthonormal basis $e_1,\dots ,e_t$ for the finite-dimensional span of the $q_i$, discarding linear dependences, and express $G_0(a)={\sum }_j{g}_j(\tilde{E}(a))e_j$. Each $g_j$ is a linear combination of the $f_i$ and hence the same-depth ${\sigma }_1$ MLP; each $e_j$ is a linear combination of the $q_i$ and hence a same-depth ${\sigma }_3$ network. If the span is zero the zero operator already suffices. Otherwise $\mathrm{\mid }g(\tilde{E}(a)){\mathrm{\mid }}_2=\mathrm{\parallel }{G}_0(a){\mathrm{\parallel }}_U<M+\gamma <2M$ on $K$.

The set $S=g(\tilde{E}(K))$ is compact with radius at most $2M$. Apply the explicitly corrected property in the Statement with $C=2M$ and tolerance $0<\eta <\epsilon -\gamma$, obtaining $b$. Compose its ${\sigma }_1$ MLP with $g$ and use final kernel $\mathrm{\mid }D{\mathrm{\mid }}^{-1}(e_j(x){)}_j$. By step 8.1 this is an operator $G_1$ of the required architecture. Orthonormality gives ${\sup }_K\mathrm{\parallel }{G}_1-G_0{\mathrm{\parallel }}_U<\eta$ and $\mathrm{\parallel }{G}_1(a){\mathrm{\parallel }}_U=\mathrm{\mid }b(g(\tilde{E}(a))){\mathrm{\mid }}_2\le 2M\le 4M$ for every $a\in A$. Thus its approximation error is below $\gamma +\eta <\epsilon$. The argument handles arbitrary positive $M$ without the source's unjustified normalization $M\ge 1$.

For the exact printed additional implication the premise ${\sigma }_1\in \mathrm{B}\mathrm{A}$ is impossible by [F5], so that implication is true but has no instances. Steps 10.1–11.1 establish the explicitly different nonvacuous branch; neither step alters the diameter-based class. This completes all asserted implications.