Physics-informed residual loss and Hölder regularization

Let $U\subset {\mathbb{R}}^d$ be a bounded Lipschitz domain, let $\mathrm{\Gamma }\subseteq \mathrm{\partial }U$ be closed, let $L$ be a differential operator on $U$, and let $B$ be a boundary operator on $\mathrm{\Gamma }$. Let $f:U\to \mathbb{R}$ and $g:\mathrm{\Gamma }\to \mathbb{R}$ be target data. For a trial function $h$, define its residuals by $r_h(x)=L[h](x)-f(x)$ and $q_h(x)=B[h](x)-g(x)$. For $0<\alpha \le 1$ and a function $v$ on a set $E$, define its Hölder seminorm by $[v{]}_{\alpha ;E}={\sup }_{x\mathrm{\ne }y\in E}\frac{\mathrm{\mid }v(x)-v(y)\mathrm{\mid }}{\mathrm{\mid }x-y{\mathrm{\mid }}^{\alpha }}\mathrm{.}$ Set $[v{]}_{\alpha ;E}=0$ when $E$ has fewer than two points. Only trial functions for which $L[h]$ on $U$ and $B[h]$ on $\mathrm{\Gamma }$ are pointwise defined and finite are admitted. For the population loss, require $f,r_h$ to be measurable with respect to $(U,{\mu }_r)$ and $g,q_h$ to be measurable with respect to $(\mathrm{\Gamma },{\mu }_b)$. For positive fixed weights ${\lambda }_r,{\lambda }_b$ and probability measures ${\mu }_r$ on $U$, ${\mu }_b$ on $\mathrm{\Gamma }$, the population loss is ${\mathrm{Loss}}^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )={\lambda }_r{\int }_U\mathrm{\mid }{r}_h{\mathrm{\mid }}^{2}d{\mu }_r+{\lambda }_b{\int }_{\mathrm{\Gamma }}\mathrm{\mid }{q}_h{\mathrm{\mid }}^{2}d{\mu }_b\mathrm{.}$ These nonnegative population integrals take values in $[0,+\mathrm{\infty }]$, so they may be infinite. The Hölder seminorms below may likewise be infinite.

For $d\ge 2$, given positive integer counts $m_r,m_b$, residual collocation points $x_r^1,\dots ,x_r^{m_r}$, and boundary collocation points $x_b^1,\dots ,x_b^{m_b}$, the empirical loss is ${\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )=\frac{{\lambda }_r}{m_r}{\sum }_{i=1}^{m_r}\mathrm{\mid }{r}_h(x_r^i){\mathrm{\mid }}^2+\frac{{\lambda }_b}{m_b}{\sum }_{j=1}^{m_b}\mathrm{\mid }{q}_h(x_b^j){\mathrm{\mid }}^2\mathrm{.}$ For $d=1$, a bounded Lipschitz domain has finite boundary, so $\mathrm{\Gamma }$ is finite. In this branch require every singleton $\{y\}\subseteq \mathrm{\Gamma }$ to be ${\mu }_b$-measurable. When every point of $\mathrm{\Gamma }$ is available deterministically, define its exact full-boundary contribution by ${\mathrm{Loss}}^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N},b}(h;{\lambda }_b)={\lambda }_b{\sum }_{y\in \mathrm{\Gamma }}{\mu }_b(\{y\})\mathrm{\mid }{q}_h(y){\mathrm{\mid }}^2={\lambda }_b{\int }_{\mathrm{\Gamma }}\mathrm{\mid }{q}_h{\mathrm{\mid }}^{2}d{\mu }_b,$ and, for a positive integer count $m_r$ and residual collocation points $x_r^1,\dots ,x_r^{m_r}$, set ${\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )=\frac{{\lambda }_r}{m_r}{\sum }_{i=1}^{m_r}\mathrm{\mid }{r}_h(x_r^i){\mathrm{\mid }}^2+{\mathrm{Loss}}^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N},b}(h;{\lambda }_b)\mathrm{.}$ For $d\ge 2$, its Hölder-regularized version is ${\mathrm{Loss}}_m(h;\lambda ,{\lambda }_m^R)={\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )+{\lambda }_{r,m}^R[L[h]{]}_{\alpha ;U}^2+{\lambda }_{b,m}^R[B[h]{]}_{\alpha ;\mathrm{\Gamma }}^2\mathrm{.}$ For $d=1$, the Hölder-regularized version is ${\mathrm{Loss}}_m(h;\lambda ,{\lambda }_m^R)={\mathrm{Loss}}_m^{\mathrm{P}\mathrm{I}\mathrm{N}\mathrm{N}}(h;\lambda )+{\lambda }_{r,m}^R[L[h]{]}_{\alpha ;U}^2;$ it uses the exact full-boundary contribution above, has no random boundary sample, and has no boundary seminorm penalty. Require the regularization coefficients appearing in either version to be nonnegative; a zero coefficient contributes zero even when its seminorm is infinite. For finite realized sample lists the empirical sums are finite; the regularized objective is allowed to take the value $+\mathrm{\infty }$. These are population and empirical functionals; they do not specify an optimization algorithm or assert that training reaches a global minimizer.

measure space

measure

integrable real and complex functions and their integrals

Shin et al., equations (2.4)–(2.6), (3.3): weighted empirical PINN loss and Hölder regularization