Neural operator between function spaces

Let $D\subset {\mathbb{R}}^d$ and $D^{\mathrm{\prime }}\subset {\mathbb{R}}^{d^{\mathrm{\prime }}}$ be bounded domains. Let $A$ and $U$ be real Banach spaces of scalar functions whose elements have continuous representatives on $\bar{D}$ and ${\bar{D}}^{\mathrm{\prime }}$, respectively, and suppose the inclusions into $C(\bar{D})$ and $C({\bar{D}}^{\mathrm{\prime }})$ are continuous when these continuous-function spaces carry the supremum norm. In the continuous-function setting, a neural operator is a composition of a pointwise lifting map, finitely many integral-kernel layers, and a pointwise projection map. A layer with channel dimensions $q_t,q_{t+1}\ge 1$ has the form $v_{t+1}(x)={\sigma }_t\text{\hspace{0.17em}⁣}(W_t{v}_t(x)+{\int }_D{\kappa }_t(x,y)v_t(y)dy+b_t(x)),x\in \bar{D},$ where $v_t:\bar{D}\to {\mathbb{R}}^{q_t}$ is continuous, $W_t\in {\mathbb{R}}^{q_{t+1}\times q_t}$, ${\kappa }_t:\bar{D}\times \bar{D}\to {\mathbb{R}}^{q_{t+1}\times q_t}$ and $b_t:\bar{D}\to {\mathbb{R}}^{q_{t+1}}$ are continuous, and the scalar activation ${\sigma }_t$ is applied coordinatewise. The lifting and projection maps are finite feedforward networks applied pointwise, with their input and output channel dimensions chosen to make the composition well-defined. All integrals are Lebesgue integrals, which exist because the integrands are continuous on a compact set. For distinct input and output domains the final integral layer has kernel on ${\bar{D}}^{\mathrm{\prime }}\times \bar{D}$, integrates over $D$, and has zero local matrix term; subsequent projection acts on ${\bar{D}}^{\mathrm{\prime }}$. For any stronger specified output space, the selected composition is required actually to map into that space.

The integral universality subclass is defined separately on $L^1(D;{\mathbb{R}}^{q_0})$, including almost-everywhere equivalence classes. Fix continuous activations ${\sigma }_1,{\sigma }_2,{\sigma }_3$ with ${\sigma }_1$ of at most linear growth, $N\ge 2$, and channel dimensions $q_1,\dots ,q_N\ge 1$. For $0\le t\le N-1$ let $S_t(v)(x)={\int }_D{\kappa }_t(x,y)v(y)dy+b_t(x),x\in \bar{D},$ where ${\kappa }_t:{\mathbb{R}}^d\times {\mathbb{R}}^d\to {\mathbb{R}}^{q_{t+1}\times q_t}$ and $b_t:{\mathbb{R}}^d\to {\mathbb{R}}^{q_{t+1}}$ are finite MLPs with hidden activation ${\sigma }_2$ and affine output. Set $v_1=S_0(a)$, $v_2=S_1(v_1)$, and $v_{t+1}=S_t({\sigma }_1(v_t))$ for $2\le t\le N-1$. An element of ${\mathrm{N}\mathrm{O}}_N$ is $G(a)(x)={\int }_D{\kappa }_N(x,y)v_N(y)dy,x\in {\bar{D}}^{\mathrm{\prime }},$ where ${\kappa }_N:{\mathbb{R}}^{d^{\mathrm{\prime }}}\times {\mathbb{R}}^d\to {\mathbb{R}}^{q_{\mathrm{o}\mathrm{u}\mathrm{t}}\times q_N}$ is a finite MLP with activation ${\sigma }_3$ and affine output. There are exactly $N-2$ intermediate ${\sigma }_1$ activations. Scalar input and output mean $q_0=q_{\mathrm{o}\mathrm{u}\mathrm{t}}=1$. The final integral is over $D$, with output variable in $D^{\mathrm{\prime }}$. This class has identity lifting and projection and zero local matrix terms. It specializes the architectural scheme with its first layer extended to $L^1$ classes; it does not require continuous input representatives.

The first integral is invariant under null changes and obeys $\mathrm{\parallel }{S}_0(a){\mathrm{\parallel }}_{\mathrm{\infty }}\le \mathrm{\parallel }{\kappa }_0{\mathrm{\parallel }}_{\mathrm{\infty }}\mathrm{\parallel }a{\mathrm{\parallel }}_1+\mathrm{\parallel }{b}_0{\mathrm{\parallel }}_{\mathrm{\infty }}$. Uniform continuity of its kernel on the compact product implies continuity of its output by the same $L^1$ estimate. All later fields are continuous and bounded, so their integrals and activations are defined; the output is continuous on ${\bar{D}}^{\mathrm{\prime }}$. Viewing it in a stronger Banach space $U$ requires the selected kernels actually to produce outputs in $U$. The universality theorem constructs such kernels; arbitrary continuous kernels alone do not guarantee this condition.

For equal input and output domains, the diagram records the composition order: a pointwise lift, integral-kernel layers on $D$, and a pointwise output projection. Each ${\mathcal{K}}_t$ box denotes the integral layer defined above; arrows denote function composition and do not assert a theorem.

This schematic is only a depiction of the composition defined above; it is not used in the discretization or universality proofs.

feedforward neural network

banach space

continuous real functions on a compact metric space

measure space

lebesgue measure and the lebesgue sigma algebra

integrable real and complex functions and their integrals

integral over a measurable set

Kovachki et al., Definition 3 and equations (13)–(17): integral-kernel neural-operator layers

Lu et al., §2: fixed-sensor branch network and output-coordinate trunk network