Neural operators and PINN consistency — Examples

Since $J(\theta )$ is a square, $J(\theta )\ge 0$, with equality exactly when ${\theta }^2=1$. Also $J(0)=1$. Hence $0$ is not a global minimizer.

By [F1], $J^{\mathrm{\prime }}(0)=0$. If ${\theta }_k=0$, the update gives ${\theta }_{k+1}=0-\eta J^{\mathrm{\prime }}(0)=0$; induction from ${\theta }_0=0$ yields ${\theta }_k=0$ for all $k$. The iterates therefore never reach either global minimizer.

Differentiating the displayed formula gives [F1]. At $x_j=j\mathrm{/}m$, $h_m^{\mathrm{\prime }}(x_j)=\sin (2\pi j)=0$. Also $h_m(0)=h_m(1)=0$ since $\cos (0)=\cos (2\pi m)=1$. Thus the empirical residual and exact boundary errors vanish.

The population residual loss is ${\int }_0^1\mathrm{\mid }{h}_m^{\mathrm{\prime }}(x){\mathrm{\mid }}^{2}dx=1\mathrm{/}2$ by [F2]. Therefore exact collocation fit at these sample points does not by itself force the population residual loss to be small.

Using $\sin (2\pi x)=(e^{2\pi ix}-e^{-2\pi ix})\mathrm{/}(2i)$ and [F1], $\widehat{v}(1)=1\mathrm{/}(2i)$, $\widehat{v}(-1)=-1\mathrm{/}(2i)$, and every other coefficient is zero.

With $R_1=R_{-1}=1$, the retained sum is $e^{2\pi ix}\mathrm{/}(2i)-e^{-2\pi ix}\mathrm{/}(2i)=v(x)$. At cutoff zero, the sum has only the $k=0$ term, which vanishes because $\widehat{v}(0)=0$.

Let $I_j=[(j-1)\mathrm{/}n,j\mathrm{/}n]$ and define ${\omega }_a(\delta )=\sup \{\mathrm{\mid }a(x)-a(y)\mathrm{\mid }:x,y\in [0,1],\mathrm{\mid }x-y\mathrm{\mid }\le \delta \}$. For each $y\in I_j$, $\mathrm{\mid }y-(j-\frac{1}{2})\mathrm{/}n\mathrm{\mid }\le 1\mathrm{/}(2n)$, so $\mid {\int }_{I_j}a(y)dy-\frac{1}{n}a((j-\frac{1}{2})\mathrm{/}n)\mid \le \frac{1}{n}{\omega }_a(1\mathrm{/}(2n))\mathrm{.}$ Summing over $j$ gives $\mathrm{\mid }G(a)(x)-G_n(a)(x)\mathrm{\mid }\le {\omega }_a(1\mathrm{/}(2n))$ for every $x$.

By [F1], ${\omega }_a(\delta )\to 0$ as $\delta \downarrow 0$. The bound in step 1.1 is independent of $x$, so $G_n(a)\to G(a)$ in $C([0,1])$. The dimension of $F_n$ changes with $n$ and its definition returns only a length-$n$ vector, so no arbitrary-point output or common function-space domain follows from $F_n$ alone.

For integers $n\ge 2$ put $t_n=3\cdot 2^{-n-2}$, $w_n=2^{-4n-4}$, $H_n=2^{2n}$ and ${\rho }_n(s)=H_n\beta ((s-t_n)\mathrm{/}{w}_n)$. Their supports are disjoint subintervals of $(0,1)$: successive centers differ by $3\cdot 2^{-n-3}$, which exceeds $w_n+w_{n+1}$, and each radius is smaller than its center. Define $\rho (s)={\sum }_{n\ge 2}{\rho }_n(s)$ for $s>0$, and $\rho (s)=0$ for $s\le 0$. On each compact subinterval of $(0,\mathrm{\infty })$ only finitely many summands are nonzero. Moreover $m_n=\int {\rho }_n\le 2H_n{w}_n=2^{-2n-3}$, so $B={\sum }_{n\ge 2}{m}_n<1$. Define $\sigma (s)={\int }_0^s\rho (t)dt$ for $s\ge 0$ and $\sigma (s)=0$ for $s<0$. The tail mass bound proves continuity at zero and uniform convergence of the primitives ${\sigma }_n(s)={\int }_0^s{\rho }_n(t)dt$; thus $\sigma =\sum {\sigma }_n$ is continuous on $\mathbb{R}$, bounded by $B$, and smooth on $(0,\mathrm{\infty })$. It is a legitimate continuous activation. It is bounded and nonconstant, hence is not polynomial even almost everywhere: continuity would make an almost-everywhere polynomial identity hold everywhere, and a bounded polynomial is constant. Both $\sigma$ and ReLU are continuous, hence Borel measurable with empty discontinuity set, and pointwise bounded on every compact interval (globally for $\sigma$, and by the interval radius for ReLU). They therefore belong to ${\mathcal{M}}_{\mathrm{p}\mathrm{t}}$ in [F3]. ReLU is not polynomial almost everywhere: continuity would give a polynomial identity everywhere, while vanishing on the negative half-line forces that polynomial to be zero, contradicting its positive values. Applying [F3] in each input dimension now places $\sigma$ in every ${\mathcal{A}}_0$ class and ReLU in every ${\mathcal{A}}_0^L$ class; ReLU is Borel and obeys $\mathrm{\mid }\mathrm{ReLU}(s)\mathrm{\mid }\le \mathrm{\mid }s\mathrm{\mid }$.

Set $\kappa (x,y)=\sigma (x+y)$ and $G(a)(x)={\int }_0^1\sigma (x+y)a(y)dy$. The kernel is exactly a one-hidden-unit network on $(x,y)$ with weights $(1,1)$, activation $\sigma$, output weight $1$, and zero biases. The other activations can be the identity. Each $G_n^{\ast }(a)(x)={\int }_0^1{\sigma }_n(x+y)a(y)dy$ is $C^1$, with derivative ${\int }_0^1{\rho }_n(x+y)a(y)dy$, and both its function norm and derivative norm are at most $m_n\mathrm{\parallel }a{\mathrm{\parallel }}_{\mathrm{\infty }}$. Consequently both the function series and derivative series converge uniformly. To verify differentiation without an interchange assumption, the finite partial sums satisfy $S_N(x)-S_N(0)={\int }_0^x{S}_N^{\mathrm{\prime }}(t)dt$; uniform convergence permits passage to the limit in this equality, and the fundamental theorem of calculus gives a continuous derivative. Hence $G:A\to U$ is bounded linear, with $\mathrm{\parallel }G(a){\mathrm{\parallel }}_U\le 2B\mathrm{\parallel }a{\mathrm{\parallel }}_{\mathrm{\infty }}$. Both $A$ and $U$ embed continuously into $C([0,1])$. To instantiate the source integral subclass as well, take $S_0$ and $S_1$ to have constant kernel $1$ and zero bias, and use $\kappa (x,y)=\sigma (x+y)$ as the final kernel, choosing ReLU for the input and intermediate kernel networks and activations. No intermediate activation occurs at depth $N=2$. This yields $\widehat{G}(a)=({\int }_0^{1}a)G(1)$, a bounded map $A\to U$. For $a=1$, both preceding integral layers and their quadratures are exactly the constant $1$, so its final continuum and quadrature outputs are precisely those analyzed below. Constant kernels are exact networks through their affine output biases.

At refinement $n$ partition $(0,1)$ into $2^n$ dyadic cells of length $h_n=2^{-n}$. Choose the first tag to be $t_n\in (0,h_n)$ and choose the midpoint of every other cell. Let $Q_n(a)(x)=h_n{\sum }_{j=1}^{2^n}\sigma (x+{\xi }_{n,j})a({\xi }_{n,j})$. These are compatible tagged Riemann sums, with mesh diameter tending to zero and the same continuum parameters at every refinement. Since every tag is positive, each summand is smooth in a neighborhood of $[0,1]$, so $Q_n(a)\in U$. For $a=1$, all derivative summands are nonnegative and $Q_n(1{)}^{\mathrm{\prime }}(0)\ge h_n\rho (t_n)=h_n{H}_n=2^n$. On the other hand, substitution in the continuum integral gives $G(1)(x)={\int }_x^{x+1}\sigma (s)ds$ and $G(1{)}^{\mathrm{\prime }}(0)=\sigma (1)-\sigma (0)=B$. Thus $\mathrm{\parallel }{Q}_n(1)-G(1){\mathrm{\parallel }}_U\ge 2^n-B\to \mathrm{\infty }$.

The set $K=\{1\}\subset A$ is compact. The continuous integrand $(x,y)\mapsto \sigma (x+y)$ on $[0,1{]}^2$ is uniformly continuous: otherwise pairs at distances tending to zero with a fixed positive function gap have convergent subsequences in the compact square and contradict continuity. Writing its modulus of continuity as $\omega$, the cellwise estimate gives $\mathrm{\parallel }{Q}_n(1)-G(1){\mathrm{\parallel }}_{\mathrm{\infty }}\le \omega (h_n)\to 0$. Thus the genuine supremum-norm conclusion coexists with divergence in the admissible output norm, refuting the asserted inference.

For the specified layer, the continuum operator and every raw quadrature map are $H(a)=H_n(a)=a$: the integral term is zero, so replacing it by any tagged sum changes nothing. In particular, an arbitrary output query still asks for $a(x)$ in the local term.

The zero function and the polynomial $a(x)={\prod }_{j=1}^N(x-{\xi }_j{)}^2$ have the same zero sample vector. The polynomial is not identically zero: any point of the interval outside the finite sampling set has strictly positive value. For $N=0$ use the empty product $a=1$. Thus their raw outputs differ by step 1.1. Any output reconstruction of the common finite vector would give them the same output by [F2], a contradiction. This disproves exact factorization of the raw rule. It asserts no obstruction to a reconstruction that only approximates the input, or to the separate compact-uniform quadrature convergence theorem.

Fix $0<\mathrm{\ell }\le 1$ and put $N=\lceil 1\mathrm{/}\mathrm{\ell }\rceil$. Since $1\mathrm{/}\mathrm{\ell }\le N<1\mathrm{/}\mathrm{\ell }+1\le 2\mathrm{/}\mathrm{\ell }$, one has $\mathrm{\ell }\mathrm{/}2\le 1\mathrm{/}N\le \mathrm{\ell }$. Partition each coordinate interval into $N$ intervals of length $1\mathrm{/}N$, assigning endpoints by a fixed half-open convention and intersecting with $U$. Their product cells lie in side-$\mathrm{\ell }$ cubes centered at their midpoints, which belong to the cells. Each has volume $N^{-D}\ge 2^{-D}{\mathrm{\ell }}^D$. On each of the $2D$ boundary faces perform the same $(D-1)$-dimensional partition. Assign shared face edges to the lowest-index incident face and then to a lowest-index neighboring cell; the grid vertices and edges have surface measure zero and lie in the closures of those cells. Each cell remains in its same closed ambient cube and its midpoint belongs to it. Its normalized surface mass is $(2D{)}^{-1}{N}^{-(D-1)}\ge (2D{)}^{-1}{2}^{-(D-1)}{\mathrm{\ell }}^{D-1}$.

Every residual ball intersection has volume at most ${\omega }_D{\rho }^D$. On each face its surface intersection is contained in a $(D-1)$-dimensional ball of radius $\rho$, of measure at most ${\omega }_{D-1}{\rho }^{D-1}$. Summing over $2D$ faces and dividing by total surface area $2D$ gives boundary probability at most ${\omega }_{D-1}{\rho }^{D-1}$. The probability densities are the positive constants $1$ and $1\mathrm{/}(2D)$ on their respective supports. These are the required upper and lower bounds with the stated small-scale range.