Neural operators and PINN consistency

Let $A$ and $U$ be real Banach spaces of functions on bounded domains $D$ and $D^{\mathrm{\prime }}$, with the input point evaluations used below defined. For every $n\ge 1$, choose finite-dimensional normed spaces $E_n,F_n$, an exactly defined sampling map $S_n:A\to E_n$ (for example, $S_n(a)=(a({\xi }_{n,1}),\dots ,a({\xi }_{n,N_n}))$ at specified input points), a finite-dimensional map ${\tilde{G}}_n:E_n\to F_n$, and an output reconstruction map $R_n:F_n\to U$. The reconstruction supplies a function with a defined value at every point of $D^{\mathrm{\prime }}$. The corresponding continuum-valued discrete map is $G_n(a):=R_n({\tilde{G}}_n(S_n(a)))\in U\mathrm{.}$ A family $(G_n{)}_{n\ge 1}$ is a discretization-invariant approximation of an operator $G:A\to U$ if its sampling maps and output reconstructions are part of the family and it approximates this single continuum operator as the input discretization is refined. In the compact-uniform version, for every compact $K\subset A$, ${\sup }_{a\in K}\mathrm{\parallel }{G}_n(a)-G(a){\mathrm{\parallel }}_U⟶0.$ For $K=\mathrm{\varnothing }$, the supremum of this nonnegative error is defined to be $0$. When $A$ consists of almost-everywhere equivalence classes, the sampling map must be explicitly defined on those classes or use a specified representative convention; point evaluation is not inferred from the equivalence class alone. This definition does not identify unrelated grid-specific maps as one operator family.

metric compactness

Kovachki et al., Definition 4 and §3.1: discretization-invariance conditions

Let $U\subset {\mathbb{R}}^d$ be a bounded Lipschitz domain, let $\mathrm{\Gamma }\subseteq \mathrm{\partial }U$ be closed, let $L$ be a differential operator on $U$, and let $B$ be a boundary operator on $\mathrm{\Gamma }$. Let $f:U\to \mathbb{R}$ and $g:\mathrm{\Gamma }\to \mathbb{R}$ be target data. For a trial function $h$, define its residuals by $r_h(x)=L[h](x)-f(x)$ and $q_h(x)=B[h](x)-g(x)$. For $0<\alpha \le 1$ and a function $v$ on a set $E$, define its Hölder seminorm by $[v{]}_{\alpha ;E}={\sup }_{x\mathrm{\ne }y\in E}\frac{\mathrm{\mid }v(x)-v(y)\mathrm{\mid }}{\mathrm{\mid }x-y{\mathrm{\mid }}^{\alpha }}\mathrm{.}$ Set $[v{]}_{\alpha ;E}=0$ when $E$ has fewer than two points. Only trial functions for which $L[h]$ on $U$ and $B[h]$ on $\mathrm{\Gamma }$ are pointwise defined and finite are admitted. For the population loss, require $f,r_h$ to be measurable with respect to $(U,{\mu }_r)$ and $g,q_h$ to be measurable with respect to $(\mathrm{\Gamma },{\mu }_b)$. For positive fixed weights ${\lambda }_r,{\lambda }_b$ and probability measures ${\mu }_r$ on $U$, ${\mu }_b$ on $\mathrm{\Gamma }$, the population loss is ${\mathrm{Loss}}^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )={\lambda }_r{\int }_U\mathrm{\mid }{r}_h{\mathrm{\mid }}^{2}d{\mu }_r+{\lambda }_b{\int }_{\mathrm{\Gamma }}\mathrm{\mid }{q}_h{\mathrm{\mid }}^{2}d{\mu }_b\mathrm{.}$ These nonnegative population integrals take values in $[0,+\mathrm{\infty }]$, so they may be infinite. The Hölder seminorms below may likewise be infinite.

For $d\ge 2$, given positive integer counts $m_r,m_b$, residual collocation points $x_r^1,\dots ,x_r^{m_r}$, and boundary collocation points $x_b^1,\dots ,x_b^{m_b}$, the empirical loss is ${\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )=\frac{{\lambda }_r}{m_r}{\sum }_{i=1}^{m_r}\mathrm{\mid }{r}_h(x_r^i){\mathrm{\mid }}^2+\frac{{\lambda }_b}{m_b}{\sum }_{j=1}^{m_b}\mathrm{\mid }{q}_h(x_b^j){\mathrm{\mid }}^2\mathrm{.}$ For $d=1$, a bounded Lipschitz domain has finite boundary, so $\mathrm{\Gamma }$ is finite. In this branch require every singleton $\{y\}\subseteq \mathrm{\Gamma }$ to be ${\mu }_b$-measurable. When every point of $\mathrm{\Gamma }$ is available deterministically, define its exact full-boundary contribution by ${\mathrm{Loss}}^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N},b}(h;{\lambda }_b)={\lambda }_b{\sum }_{y\in \mathrm{\Gamma }}{\mu }_b(\{y\})\mathrm{\mid }{q}_h(y){\mathrm{\mid }}^2={\lambda }_b{\int }_{\mathrm{\Gamma }}\mathrm{\mid }{q}_h{\mathrm{\mid }}^{2}d{\mu }_b,$ and, for a positive integer count $m_r$ and residual collocation points $x_r^1,\dots ,x_r^{m_r}$, set ${\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )=\frac{{\lambda }_r}{m_r}{\sum }_{i=1}^{m_r}\mathrm{\mid }{r}_h(x_r^i){\mathrm{\mid }}^2+{\mathrm{Loss}}^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N},b}(h;{\lambda }_b)\mathrm{.}$ For $d\ge 2$, its Hölder-regularized version is ${\mathrm{Loss}}_m(h;\lambda ,{\lambda }_m^R)={\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )+{\lambda }_{r,m}^R[L[h]{]}_{\alpha ;U}^2+{\lambda }_{b,m}^R[B[h]{]}_{\alpha ;\mathrm{\Gamma }}^2\mathrm{.}$ For $d=1$, the Hölder-regularized version is ${\mathrm{Loss}}_m(h;\lambda ,{\lambda }_m^R)={\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )+{\lambda }_{r,m}^R[L[h]{]}_{\alpha ;U}^2;$ it uses the exact full-boundary contribution above, has no random boundary sample, and has no boundary seminorm penalty. Require the regularization coefficients appearing in either version to be nonnegative; a zero coefficient contributes zero even when its seminorm is infinite. For finite realized sample lists the empirical sums are finite; the regularized objective is allowed to take the value $+\mathrm{\infty }$. These are population and empirical functionals; they do not specify an optimization algorithm or assert that training reaches a global minimizer.

measure space

measure

integrable real and complex functions and their integrals

Shin et al., equations (2.4)–(2.6), (3.3): weighted empirical PINN loss and Hölder regularization

Let $U\subset {\mathbb{R}}^d$ be open. For finite real coefficient functions $a_{ij},b_i,c_i,d$ on $U$, the positive divergence-form expression is $L[u]={\sum }_{i,j=1}^d{D}_i(a_{ij}{D}_{j}u)+{\sum }_{i=1}^d{D}_i(b_{i}u)+{\sum }_{i=1}^d{c}_i{D}_{i}u+du\mathrm{.}$ Its classical domain consists of those $u\in C^2(U)$ for which every displayed flux derivative $D_i(a_{ij}{D}_{j}u)$ and $D_i(b_{i}u)$ exists pointwise and is finite on $U$. The classical formula assigns no value outside this domain. In particular, if each $D_i{a}_{ij}$ and $D_i{b}_i$ exists pointwise and is finite, the product rule makes every $u\in C^2(U)$ admissible and gives the nondivergence expansion $L[u]={\sum }_{i,j=1}^d{a}_{ij}{D}_i{D}_{j}u+{\sum }_{j=1}^d{\tilde{b}}_j{D}_{j}u+\tilde{c}u,$ with ${\tilde{b}}_j={\sum }_{i=1}^d{D}_i{a}_{ij}+b_j+c_j,\tilde{c}={\sum }_{i=1}^d{D}_i{b}_i+d\mathrm{.}$ The positive divergence convention here means that the principal part is $+{\sum }_{i,j}{D}_i(a_{ij}{D}_{j}u)$. This sign is part of the definition.

When a weak formulation is used, its divergence derivatives are instead distributional derivatives of locally integrable fluxes, with the integrability hypotheses supplied in that formulation. Equality with a pointwise expression requires separate justification; pointwise differentiability alone does not assert it.

ck and multi index notation in several variables

Shin et al., equation (3.6): elliptic divergence-form operator and expanded coefficients

The source displays ${\sum }_{i,j}{D}_i(a_{ij}{D}_{j}u+b_{i}u)$ while its expanded coefficients and energy argument count $D_i(b_{i}u)$ only once. Read literally, that displayed sum instead gives $d{\sum }_i{D}_i(b_{i}u)$ and changes both the drift and potential. The active definition above separates the two sums explicitly and agrees with the stated expanded coefficients. The original printed expression is retained here as a source discrepancy, not used as an algebraic identity.

Let $U\subset {\mathbb{R}}^d$ be nonempty, bounded, and open, let $T>0$, and set $Q=U\times (0,T)$ and ${\mathrm{\Gamma }}_T=(\bar{U}\times \{0\})\cup (\mathrm{\partial }U\times [0,T])\mathrm{.}$ The final-time face $U\times \{T\}$ is excluded from ${\mathrm{\Gamma }}_T$. For time-dependent coefficients define $L_t[u]={\sum }_{i,j=1}^d{D}_i(a_{ij}(x,t)D_{j}u)+{\sum }_{i=1}^d{D}_i(b_i(x,t)u)+{\sum }_{i=1}^d{c}_i(x,t)D_{i}u+d(x,t)u\mathrm{.}$ Where the coefficient derivatives exist, write its nondivergence expansion as $L_t[u]={\sum }_{i,j}{a}_{ij}{D}_i{D}_{j}u+{\sum }_i{\tilde{b}}_i{D}_{i}u+\tilde{c}u,$ where ${\tilde{b}}_i={\sum }_j{D}_j{a}_{ji}+b_i+c_i$ and $\tilde{c}={\sum }_i{D}_i{b}_i+d$. For symmetric $a$, the drift also equals ${\sum }_j{D}_j{a}_{ij}+b_i+c_i$, the form used by the symmetric source consumers. The initial-boundary value problem uses the sign convention $-u_t+L_t[u]=f\text{in }Q,u=\phi \text{on }\mathrm{\partial }U\times [0,T],u=g\text{on }\bar{U}\times \{0\},$ with compatibility $g(x)=\phi (x,0)$ on $\mathrm{\partial }U$. Its residual for a trial function $h$ is $f+h_t-L_t[h]$, and its prescribed-data error is measured on ${\mathrm{\Gamma }}_T$.

For the source notation $C_{2+\alpha }(Q)$, use the parabolic distance $d_p((x,t),(y,s))=(\mathrm{\mid }x-y{\mathrm{\mid }}^2+\mathrm{\mid }t-s\mathrm{\mid }{)}^{1\mathrm{/}2}\mathrm{.}$ Let $B=U\times \{0\}$ and $S_{\tau }=\mathrm{\partial }U\times [0,\tau ]$. For $P=(x,\tau )\in Q$, set $\delta (P)={\inf }_{R\in B\cup S_{\tau }}{d}_p(P,R)$ and ${\delta }_{P,Q}=\min \{\delta (P),\delta (Q)\}$. For an integer $m\ge 0$, define $\mathrm{\mid }v{\mathrm{\mid }}_{m,\alpha }={\sup }_{P\in Q}\delta (P{)}^m\mathrm{\mid }v(P)\mathrm{\mid }+{\sup }_{P\mathrm{\ne }Q\in Q}{\delta }_{P,Q}^{m+\alpha }\frac{\mathrm{\mid }v(P)-v(Q)\mathrm{\mid }}{d_p(P,Q{)}^{\alpha }}\mathrm{.}$ The class $C_{2+\alpha }(Q)$ consists of functions for which $\mathrm{\mid }u{\mathrm{\mid }}_{0,\alpha }+{\sum }_i\mathrm{\mid }{D}_{i}u{\mathrm{\mid }}_{1,\alpha }+{\sum }_{i,j}\mathrm{\mid }{D}_{ij}u{\mathrm{\mid }}_{1,\alpha }+\mathrm{\mid }{u}_t{\mathrm{\mid }}_{2,\alpha }<\mathrm{\infty },$ with the indicated derivatives continuous in $Q$. This fixes the source's weighted parabolic Hölder convention.

Shin et al., equations (2.1)–(2.3) and Assumption 3.4: parabolic sign convention, data, and coefficients

Published parabolic-cylinder definition: initial and lateral boundary, excluding the final-time face

The source displays ${\sum }_{i,j}{D}_i(a_{ij}{D}_{j}u+b_{i}u)$ while its expanded coefficients and energy argument count $D_i(b_{i}u)$ only once. Read literally, that displayed sum instead gives $d{\sum }_i{D}_i(b_{i}u)$ and changes both the drift and potential. The active definition above separates the two sums explicitly and agrees with the stated expanded coefficients. The original printed expression is retained here as a source discrepancy, not used as an algebraic identity.

Let $d\ge 1$, $s>0$, and let $X\subset {\mathbb{R}}^d$ carry a probability measure $\mu$. Fix an integer $m\ge 1$ and $c>0$, and put $\epsilon =(c\sqrt{m}{)}^{-1\mathrm{/}s}$. Suppose there is a finite measurable partition $X_1,\dots ,X_K$ of $X$ at this particular scale such that each cell lies in a cube of side $\epsilon$ and $\mu (X_j)\ge c{\epsilon }^s$. Let $Y_1,\dots ,Y_m$ be independent random points each with law $\mu$. Then with probability at least $1-\sqrt{m}{(1-\frac{1}{\sqrt{m}})}^m$ every point of $X$ lies within Euclidean distance $\sqrt{d}{c}^{-1\mathrm{/}s}{m}^{-1\mathrm{/}(2s)}$ of at least one sample point. For an interior space of dimension $d$, use $s=d$; for a boundary space of Hausdorff dimension $d-1>0$, use $s=d-1$ and its own cell-mass constant. If $d=1$ and every boundary point is available deterministically, the boundary fill distance is zero and no random boundary sample is needed.

probability measure

independent random elements

Shin et al., Appendix B, Lemma B.2: probabilistic space-filling estimate

Shin et al., Appendix B, proof of Lemma B.2

It suffices to assume the partitions exist for $0<\epsilon \le {\epsilon }_0$ and then restrict to $m\ge \max \{1,c^{-2}{\epsilon }_0^{-2s}\}$. Indeed, this is exactly the condition $(c\sqrt{m}{)}^{-1\mathrm{/}s}\le {\epsilon }_0$. For upper ball-mass bounds used at the fill radius, also require $\sqrt{D}(c\sqrt{m}{)}^{-1\mathrm{/}s}\le {\rho }_0$ if their range is $0<\rho \le {\rho }_0$. The normalized Appendix B mass is $c{\epsilon }^s=1\mathrm{/}\sqrt{m}$, so its failure probability is $\sqrt{m}(1-1\mathrm{/}\sqrt{m}{)}^m$, independently of $c$. If one deliberately chooses $0<c\le 1$, this is no larger than $\sqrt{m}(1-c\mathrm{/}\sqrt{m}{)}^m$, recovering the weaker later printed probability with those same chosen constants. Replacing a lower-mass constant by a smaller one is valid, but all radii, $C_m$, and regularization thresholds must then use the new constant. No such replacement is made implicitly.

Let $X$ be a nonempty measurable space, let $d\ge 1$, let $c>0$, and let $\mu$ be a probability measure on $X$. There is no family of finite measurable partitions ${\mathcal{P}}_{\epsilon }$ of $X$, one for every $\epsilon >0$, for which every cell $E\in {\mathcal{P}}_{\epsilon }$ satisfies $\mu (E)\ge c{\epsilon }^d$.

Shin et al., Assumption 3.1, clause 3: the all-$ϵ>0$ lower cell-mass condition

There exists $b\in C_c^{\mathrm{\infty }}(\mathbb{R})$ with $0\le b\le 1$, $b(x)=1$ for $\mathrm{\mid }x\mathrm{\mid }\le 1$, and $b(x)=0$ for $\mathrm{\mid }x\mathrm{\mid }\ge 2$, such that the profile $\chi :[0,\mathrm{\infty })\to \mathbb{R}$ defined by $\chi (r)=b(4r)$ is the restriction of a smooth function on $\mathbb{R}$, equals $1$ for $0\le r\le 1\mathrm{/}4$, and equals $0$ for $r\ge 1\mathrm{/}2$.

Here $C_c^{\mathrm{\infty }}(\mathbb{R})$ denotes the infinitely differentiable, compactly supported real-valued functions.

Let $(X,d_X)$ and $(Y,d_Y)$ be metric spaces, let $K\subseteq X$ be compact, and let $f,f_n:X\to Y$ be continuous for every integer $n\ge 1$. Suppose that for every $\epsilon >0$ there is $N\ge 1$ such that $d_Y(f_n(x),f(x))<\epsilon$ for all $n\ge N$ and all $x\in K$. Then $W=f[K]\cup {\bigcup }_{n=1}^{\mathrm{\infty }}{f}_n[K]$ is compact in $Y$. The empty-set case is included, and the proof uses no choice principle.

Assume Countable Choice. Let $D\subset {\mathbb{R}}^d$ be a bounded Lipschitz domain. Each of the real spaces $L^p(D)$ for $1\le p<\mathrm{\infty }$, $W^{m,p}(D)$ for $m\in \mathbb{N}$ and $1\le p<\mathrm{\infty }$, and $C^m(\bar{D})$ for $m\in {\mathbb{N}}_0$ has the approximation property: on every norm-compact subset, the identity can be approximated uniformly by bounded finite-rank linear maps. Here the $C^m$ norm is the maximum of the uniform norms of the derivatives through order $m$, extended continuously to $\bar{D}$. For any bounded Lipschitz domain $D^{\mathrm{\prime }}\subset {\mathbb{R}}^{d^{\mathrm{\prime }}}$, the corresponding real output spaces $L^{p_2}(D^{\mathrm{\prime }})$ for $1\le p_2<\mathrm{\infty }$, $W^{m_2,p_2}(D^{\mathrm{\prime }})$ for $m_2\in \mathbb{N}$ and $1\le p_2<\mathrm{\infty }$, and $C^{m_2}(\overline{D^{\mathrm{\prime }}})$ for $m_2\in {\mathbb{N}}_0$ also have the approximation property.

The Axiom of Countable Choice (${\mathrm{A}\mathrm{C}}_{\omega }$)

Approximation property and bounded approximation property

Kovachki et al., Lemma 26

Inputs are a dimension $d\ge 1$, the data and operators specified in the physics-informed residual loss definition, $0<\alpha \le 1$, positive weights ${\lambda }_r,{\lambda }_b$, and an admissible trial function $h$. The procedure returns the scalar empirical objective value $J_m(h)$. It uses the realized sample points and target values as inputs; it does not generate random samples.

For $d\ge 2$, also supply positive integer counts $m_r,m_b$, residual samples $x_r^1,\dots ,x_r^{m_r}\in U$ with their observed values $f(x_r^i)$, boundary samples $x_b^1,\dots ,x_b^{m_b}\in \mathrm{\Gamma }$ with their observed values $g(x_b^j)$, constants $c_r,c_b,C_r,C_b>0$, and small-scale cutoffs ${\mathrm{\ell }}_{0,i},{\rho }_{0,i}>0$. At the selected partition scales $0<{\mathrm{\ell }}_i\le {\mathrm{\ell }}_{0,i}$, require cells $U_j^{{\mathrm{\ell }}_r}$ and ${\mathrm{\Gamma }}_j^{{\mathrm{\ell }}_b}$ with masses at least $c_r{\mathrm{\ell }}_r^d$ and $c_b{\mathrm{\ell }}_b^{d-1}$, each contained in a cube of side ${\mathrm{\ell }}_i$. Require upper ball-mass bounds ${\mu }_r({\bar{B}}_{\rho }(x)\cap U)\le C_r{\rho }^d$ and ${\mu }_b({\bar{B}}_{\rho }(y)\cap \mathrm{\Gamma })\le C_b{\rho }^{d-1}$ for every center and for all $0<\rho \le {\rho }_{0,i}$. The admissible counts satisfy, for $i\in \{r,b\}$, $m_i\ge \max \{1,c_i^{-2}{\mathrm{\ell }}_{0,i}^{-2s_i},c_i^{-2}{d}^{s_i}{\rho }_{0,i}^{-2s_i}\},$ so the computed partition scales and fill radii lie in the specified ranges.

For $d=1$, supply a positive integer $m_r$, residual samples $x_r^1,\dots ,x_r^{m_r}\in U$ with observed values $f(x_r^i)$, constants $c_r,C_r>0$, and cutoffs ${\mathrm{\ell }}_{0,r},{\rho }_{0,r}>0$. At the residual scale ${\mathrm{\ell }}_r$ below, require a finite measurable partition with each cell contained in an interval of length ${\mathrm{\ell }}_r$ and mass at least $c_r{\mathrm{\ell }}_r$, and require upper ball-mass bounds ${\mu }_r({\bar{B}}_{\rho }(x)\cap U)\le C_r\rho$ for every center $x\in U$ and all $0<\rho \le {\rho }_{0,r}$. The admissible count satisfies $m_r\ge \max \{1,c_r^{-2}{\mathrm{\ell }}_{0,r}^{-2},c_r^{-2}{\rho }_{0,r}^{-2}\},$ so ${\mathrm{\ell }}_r={\delta }_r$ is in the specified ranges. Supply the observed boundary values at every point of the finite set $\mathrm{\Gamma }$ with their masses ${\mu }_b(\{y\})$. No random boundary sample or boundary sample count is used.

For $d\ge 2$, set $s_r=d$, $s_b=d-1$, and, for $i\in \{r,b\}$, compute ${\mathrm{\ell }}_i=(c_i\sqrt{m_i}{)}^{-1\mathrm{/}{s}_i},{\delta }_i=\sqrt{d}{\mathrm{\ell }}_i,{\kappa }_i=C_i\mathrm{/}{c}_i,$ $C_m=3{\max }_{i\in \{r,b\}}\{{\kappa }_i{d}^{s_i\mathrm{/}2}{m}_i^{1\mathrm{/}2}\},{\widehat{\lambda }}_{i,m}^R=\frac{3{\lambda }_i{\delta }_i^{2\alpha }}{C_m}\mathrm{.}$ For $d=1$, compute ${\mathrm{\ell }}_r={\delta }_r=(c_r\sqrt{m_r}{)}^{-1},{\kappa }_r=C_r\mathrm{/}{c}_r,C_m=3{\kappa }_r{m}_r^{1\mathrm{/}2},$ ${\widehat{\lambda }}_{r,m}^R=\frac{3{\lambda }_r{\delta }_r^{2\alpha }}{C_m}=\frac{{\lambda }_r{c}_r^{-2\alpha }}{{\kappa }_r}{m}_r^{-\alpha -1\mathrm{/}2},{\widehat{\lambda }}_{b,m}^R=0.$ For $d\ge 2$, use supplied regularization weights ${\lambda }_{i,m}^R$ satisfying ${\lambda }_{i,m}^R\ge {\widehat{\lambda }}_{i,m}^R$ componentwise; for a sequence of sample counts also require $\mathrm{\parallel }{\lambda }_m^R{\mathrm{\parallel }}_{\mathrm{\infty }}=O(\mathrm{\parallel }{\widehat{\lambda }}_m^R{\mathrm{\parallel }}_{\mathrm{\infty }})$. For $d=1$, set ${\lambda }_{b,m}^R=0$ and choose ${\lambda }_{r,m}^R\ge {\widehat{\lambda }}_{r,m}^R$, with the corresponding one-coordinate max-norm order condition across a sequence of counts. If no residual weights are supplied, set them equal to their thresholds.

Finally evaluate and return $J_m(h)={\mathrm{Loss}}_m(h;\lambda ,{\lambda }_m^R)$ using the realized residual values, the realized boundary values when $d\ge 2$, the exact weighted full-boundary contribution when $d=1$, and the residual seminorm penalty when $d=1$ or both seminorm penalties when $d\ge 2$. The regularization thresholds are the source's exact thresholds and cannot be replaced by a generic positive penalty.

physics informed neural network loss

Shin et al., equations (3.1)–(3.3): exact thresholds, scaling condition, and objective

For $d\ge 2$, the exact residual and boundary threshold coordinates are ${\widehat{\lambda }}_{r,m}^R=\frac{3{\lambda }_r(\sqrt{d}{)}^{2\alpha }{c}_r^{-2\alpha \mathrm{/}d}{m}_r^{-\alpha \mathrm{/}d}}{C_m},{\widehat{\lambda }}_{b,m}^R=\frac{3{\lambda }_b(\sqrt{d}{)}^{2\alpha }{c}_b^{-2\alpha \mathrm{/}(d-1)}{m}_b^{-\alpha \mathrm{/}(d-1)}}{C_m},$ where $C_m=3\max \{(C_r\mathrm{/}{c}_r)(\sqrt{d}{)}^d{m}_r^{1\mathrm{/}2},(C_b\mathrm{/}{c}_b)(\sqrt{d}{)}^{d-1}{m}_b^{1\mathrm{/}2}\}$. For $d=1$, the residual threshold is ${\widehat{\lambda }}_{r,m}^R=({\lambda }_r{c}_r^{-2\alpha }\mathrm{/}{\kappa }_r)m_r^{-\alpha -1\mathrm{/}2}$, with ${\kappa }_r=C_r\mathrm{/}{c}_r$, and the boundary threshold is zero. The weight vector must satisfy the componentwise lower bound and max-norm order condition in the Algorithm.

The original all-$\epsilon >0$ sampling quantifier has no probability-measure instances; it is preserved and refuted in the all-scale inconsistency lemma. The small-scale range above is the corrected active sampling convention. No lower-mass constant normalization is implicit in this algorithm.

Let $D\subset {\mathbb{R}}^d$ and $D^{\mathrm{\prime }}\subset {\mathbb{R}}^{d^{\mathrm{\prime }}}$ be bounded domains. Let $A$ and $U$ be real Banach spaces of scalar functions whose elements have continuous representatives on $\bar{D}$ and ${\bar{D}}^{\mathrm{\prime }}$, respectively, and suppose the inclusions into $C(\bar{D})$ and $C({\bar{D}}^{\mathrm{\prime }})$ are continuous when these continuous-function spaces carry the supremum norm. In the continuous-function setting, a neural operator is a composition of a pointwise lifting map, finitely many integral-kernel layers, and a pointwise projection map. A layer with channel dimensions $q_t,q_{t+1}\ge 1$ has the form $v_{t+1}(x)={\sigma }_t\text{\hspace{0.17em}⁣}(W_t{v}_t(x)+{\int }_D{\kappa }_t(x,y)v_t(y)dy+b_t(x)),x\in \bar{D},$ where $v_t:\bar{D}\to {\mathbb{R}}^{q_t}$ is continuous, $W_t\in {\mathbb{R}}^{q_{t+1}\times q_t}$, ${\kappa }_t:\bar{D}\times \bar{D}\to {\mathbb{R}}^{q_{t+1}\times q_t}$ and $b_t:\bar{D}\to {\mathbb{R}}^{q_{t+1}}$ are continuous, and the scalar activation ${\sigma }_t$ is applied coordinatewise. The lifting and projection maps are finite feedforward networks applied pointwise, with their input and output channel dimensions chosen to make the composition well-defined. All integrals are Lebesgue integrals, which exist because the integrands are continuous on a compact set. For distinct input and output domains the final integral layer has kernel on ${\bar{D}}^{\mathrm{\prime }}\times \bar{D}$, integrates over $D$, and has zero local matrix term; subsequent projection acts on ${\bar{D}}^{\mathrm{\prime }}$. For any stronger specified output space, the selected composition is required actually to map into that space.

The integral universality subclass is defined separately on $L^1(D;{\mathbb{R}}^{q_0})$, including almost-everywhere equivalence classes. Fix continuous activations ${\sigma }_1,{\sigma }_2,{\sigma }_3$ with ${\sigma }_1$ of at most linear growth, $N\ge 2$, and channel dimensions $q_1,\dots ,q_N\ge 1$. For $0\le t\le N-1$ let $S_t(v)(x)={\int }_D{\kappa }_t(x,y)v(y)dy+b_t(x),x\in \bar{D},$ where ${\kappa }_t:{\mathbb{R}}^d\times {\mathbb{R}}^d\to {\mathbb{R}}^{q_{t+1}\times q_t}$ and $b_t:{\mathbb{R}}^d\to {\mathbb{R}}^{q_{t+1}}$ are finite MLPs with hidden activation ${\sigma }_2$ and affine output. Set $v_1=S_0(a)$, $v_2=S_1(v_1)$, and $v_{t+1}=S_t({\sigma }_1(v_t))$ for $2\le t\le N-1$. An element of ${\mathrm{N}\mathrm{O}}_N$ is $G(a)(x)={\int }_D{\kappa }_N(x,y)v_N(y)dy,x\in {\bar{D}}^{\mathrm{\prime }},$ where ${\kappa }_N:{\mathbb{R}}^{d^{\mathrm{\prime }}}\times {\mathbb{R}}^d\to {\mathbb{R}}^{q_{\mathrm{o}\mathrm{u}\mathrm{t}}\times q_N}$ is a finite MLP with activation ${\sigma }_3$ and affine output. There are exactly $N-2$ intermediate ${\sigma }_1$ activations. Scalar input and output mean $q_0=q_{\mathrm{o}\mathrm{u}\mathrm{t}}=1$. The final integral is over $D$, with output variable in $D^{\mathrm{\prime }}$. This class has identity lifting and projection and zero local matrix terms. It specializes the architectural scheme with its first layer extended to $L^1$ classes; it does not require continuous input representatives.