Open frontiers in calibrated and robust learned inversion — Examples and Counterexamples

Let $n\ge 1$, $\sigma ,\tau >0$, and $0<\gamma <1$. Given a scalar truth $\theta \in \mathbb{R}$, observe independent $Y_i\sim N(\theta ,{\sigma }^2)$, $i=1,\dots ,n$, and use the proper prior $\theta \sim N(0,{\tau }^2)$. Write $\bar{Y}=n^{-1}{\sum }_i{Y}_i$, $v_n=({\tau }^{-2}+n{\sigma }^{-2}{)}^{-1}$, $a_n=v_{n}n{\sigma }^{-2}$, and $z_{\gamma }={\mathrm{\Phi }}^{-1}(1-\gamma \mathrm{/}2)$, where $\mathrm{\Phi }$ is the standard normal cumulative distribution function. The exact posterior is $N(a_n\bar{Y},v_n)$ and its central $(1-\gamma )$ credible interval is $C_n(\bar{Y})=[a_n\bar{Y}-z_{\gamma }\sqrt{v_n},a_n\bar{Y}+z_{\gamma }\sqrt{v_n}]$. Its frequentist coverage at truth $\theta$ is $P_{\theta }\{\theta \in C_n(\bar{Y})\}=\mathrm{\Phi }\text{\hspace{0.17em}⁣}\left(\frac{z_{\gamma }\sqrt{v_n}+(1-a_n)\theta }{a_n\sigma \mathrm{/}\sqrt{n}}\right)-\mathrm{\Phi }\text{\hspace{0.17em}⁣}\left(\frac{-z_{\gamma }\sqrt{v_n}+(1-a_n)\theta }{a_n\sigma \mathrm{/}\sqrt{n}}\right)\mathrm{.}$

The calibration frontier whose exact-computation baseline is isolated here. ‡

Fix $n\ge 1$, $\sigma ,\tau >0$, and $0<\gamma <1$. Observe independent $Y_i\sim N(\theta ,{\sigma }^2)$ under truth $\theta \in \mathbb{R}$ and use prior $N(0,{\tau }^2)$. Let $v_n=({\tau }^{-2}+n{\sigma }^{-2}{)}^{-1}$, $a_n=v_{n}n{\sigma }^{-2}$, and $z_{\gamma }={\mathrm{\Phi }}^{-1}(1-\gamma \mathrm{/}2)$, where $\mathrm{\Phi }$ is the standard normal cumulative distribution function. The exact posterior credible interval is $[a_n\bar{Y}-z_{\gamma }\sqrt{v_n},a_n\bar{Y}+z_{\gamma }\sqrt{v_n}]$. Since $0<a_n<1$, its repeated-sampling coverage at fixed $n$ tends to zero as $\mathrm{\mid }\theta \mathrm{\mid }\to \mathrm{\infty }$, despite exact posterior computation and exact sampling.

The calibrated-inversion question and its exact Bayesian baseline. ‡

Fix $R,a_0,\sigma >0$ and $0\le {\rho }_A<a_0\mathrm{/}2$. The working inverse is $g(y)=y\mathrm{/}{a}_0$. Under a test operator $a=a_0+\delta$ with $\mathrm{\mid }\delta \mathrm{\mid }\le {\rho }_A$, the observation is $Y=a\theta +\epsilon$ for $\theta \in [-R,R]$ and independent $\epsilon \sim N(0,{\sigma }^2)$. For any two truth laws ${\mu }_0,\mu$ on $[-R,R]$, define $W_1(\mu ,{\mu }_0)$ as the infimum of $E\mathrm{\mid }\mathrm{\Theta }-{\mathrm{\Theta }}_0\mathrm{\mid }$ over their couplings. The risk is $R_{\mu ,\delta }(g)=E_{\theta \sim \mu ,\epsilon }\mathrm{\mid }g((a_0+\delta )\theta +\epsilon )-\theta {\mathrm{\mid }}^2=\frac{{\delta }^2}{a_0^2}{E}_{\mu }{\theta }^2+\frac{{\sigma }^2}{a_0^2},$ and it satisfies $R_{\mu ,\delta }(g)\le ({\rho }_A^2{R}^2+{\sigma }^2)\mathrm{/}{a}_0^2$ and $\mathrm{\mid }{R}_{\mu ,\delta }(g)-R_{{\mu }_0,\delta }(g)\mathrm{\mid }\le \frac{2{\delta }^{2}R}{a_0^2}{W}_1(\mu ,{\mu }_0)\mathrm{.}$

The bounded operator and test-law shift frontier illustrated by a conditioned scalar inverse.

Fix a truth bound $R>0$. For every absolute operator-shift radius ${\rho }_A>0$, take the working scalar operator $A_0={\rho }_A\mathrm{/}2$, the true operator $A=0$, the noiseless datum $Y=A\theta$ at truth $\theta =R$, and the working inverse $g(y)=y\mathrm{/}{A}_0$. Then $\mathrm{\mid }A-A_0\mathrm{\mid }={\rho }_A\mathrm{/}2\le {\rho }_A$ but $\mathrm{\mid }g(Y)-\theta {\mathrm{\mid }}^2=R^2$. Thus no recovery modulus tending to zero with the absolute operator error can hold uniformly when the baseline inverse conditioning is allowed to vanish.

The robust-inversion frontier and its need for explicit conditioning hypotheses.

Let $b>0$, $0<w<1$, and let the two-mode posterior be ${\pi }_w=w{\delta }_{-b}+(1-w){\delta }_b$ on $\mathbb{R}$. From $M\ge 1$ independent exact posterior draws $X_1,\dots ,X_M$, define $\widehat{w}=M^{-1}{\sum }_i\mathbf{1}\{X_i=-b\}$ and ${\widehat{\pi }}_M=\widehat{w}{\delta }_{-b}+(1-\widehat{w}){\delta }_b$. The one-Wasserstein error and the probability that at least one posterior mode is absent from the sample are $W_1({\widehat{\pi }}_M,{\pi }_w)=2b\mathrm{\mid }\widehat{w}-w\mathrm{\mid },P\{\widehat{w}\in \{0,1\}\}=(1-w{)}^M+w^M,$ and $E{W}_1({\widehat{\pi }}_M,{\pi }_w)\le 2b\sqrt{w(1-w)\mathrm{/}M}$.

The finite-compute posterior-fidelity question, with score and time-discretization errors set to zero.

For the two-mode posterior ${\pi }_w=w{\delta }_{-b}+(1-w){\delta }_b$ with $b>0$ and $0<w<1\mathrm{/}2$, $M\ge 1$ independent exact draws miss the rare mode $-b$ with probability $(1-w{)}^M$. For any fixed $M$, this probability tends to one as $w\downarrow 0$. If instead $w\in [w_{\min },1-w_{\min }]$ with $0<w_{\min }\le 1\mathrm{/}2$, then the probability of missing either mode is at most $2(1-w_{\min }{)}^M$, so $M\ge \log (2\mathrm{/}\alpha )\mathrm{/}[-\log (1-w_{\min })]$ suffices to make it at most $\alpha \in (0,1)$.

A positive lower bound on posterior mode mass is one condition needed for uniform finite-budget guarantees.

Write ${\mathrm{\ell }}^2={\mathrm{\ell }}^2({\mathbb{N}}_{>0};\mathbb{R})$, where ${\mathbb{N}}_{>0}=\{1,2,\dots \}$, so all sequence coordinates below have positive indices. Let $p,s,R>0$, let $m\ge 1$ be an integer, let $\theta \in {\mathrm{\ell }}^2$ satisfy ${\sum }_{j\ge 1}{j}^{2s}{\theta }_j^2\le R^2$, and let $(A_p\theta {)}_j=j^{-p}{\theta }_j$. Given data $y=A_p\theta +\eta$ with $\mathrm{\parallel }\eta {\mathrm{\parallel }}_{{\mathrm{\ell }}^2}\le \delta$, the mesh-$m$ truncated inverse is $g_m(y)=(j^p{y}_j{)}_{j\le m}$ with zero output coordinates for $j>m$. It satisfies the continuum error bound $\mathrm{\parallel }{g}_m(y)-\theta {\mathrm{\parallel }}_{{\mathrm{\ell }}^2}\le m^p\delta +R(m+1{)}^{-s}\mathrm{.}$ This separates measurement-noise amplification from the mesh truncation tail; it makes no statistical or neural-operator claim.

A mesh-transfer example on the compact diagonal inverse-map domain. ‡

Let $N\ge 1$ and let $x_1,\dots ,x_N\in [0,1]$ be arbitrary training inputs. For every deterministic learning rule that returns a prediction function $g:[0,1]\to \mathbb{R}$ from the observed pairs $(x_i,f(x_i))$, there are two $1$-Lipschitz functions $f_0,f_1:[0,1]\to \mathbb{R}$ agreeing with each other and taking the common value $0$ at every training input but satisfying $\mathrm{\parallel }{f}_1-f_0{\mathrm{\parallel }}_{\mathrm{\infty }}\ge \frac{1}{2(N+1)}\mathrm{.}$ Consequently, at the midpoint of a largest sample-free interval, the returned prediction function $g$ has absolute prediction error at least $1\mathrm{/}(4(N+1))$ for at least one of these two indistinguishable target functions.

A finite-sample limit that motivates explicit inverse-map and architecture classes. ‡

In the finite-dimensional Gaussian credible-ball question reproduced below, choose $d=n=1$, $R=1$, $\sigma =1$, $T=1$, $\beta (t)=1$, and $\gamma =1\mathrm{/}4$, with arbitrary positive $p,s,a$. Take the numerical operator equal to the true operator, so ${\delta }_A=0$. For every integer $K\ge 1$ there is a jointly Borel finite score field with integrated score error exactly zero, whose Euler terminal law is $N(1,K^{-1})$ for every observation. With $M=K$ independent terminal draws, the resulting empirical ball has frequentist coverage tending to zero at truth $\theta =0$, although the stated uniform boundary-margin condition holds with exponent $\kappa =1$. The exact credible-ball baseline satisfies ${\delta }_{1,1\mathrm{/}4}^0<3\mathrm{/}4$. Consequently no bound in that question can have $\mathrm{\Phi }\to 0$ uniformly over its original score class as $({\delta }_A,{\epsilon }_s,K^{-1},M^{-1\mathrm{/}2})\to (0,0,0,0)$. The original formulation is retained verbatim under Original question.

Knapik, van der Vaart, and van Zanten, full paper

Szabó, van der Vaart, and van Zanten, full paper

Giné and Nickl, full textbook

Narnhofer et al., full paper

Fix integers $d,n\ge 1$, parameters $p,s,a,R,\sigma >0$, a credibility error $\gamma \in (0,1)$, and a continuous schedule $\beta :[0,T]\to (0,\mathrm{\infty })$ with $T>0$. Let ${\mathrm{\Theta }}_s(R)=\{\theta \in {\mathbb{R}}^d:{\sum }_{j=1}^d{j}^{2s}{\theta }_j^2\le R^2\},A_p=\mathrm{diag}(1^{-p},2^{-p},\dots ,d^{-p})\mathrm{.}$ Observe $Y_i=A_p\theta +{\epsilon }_i$, $1\le i\le n$, with independent ${\epsilon }_i\sim N(0,{\sigma }^2{I}_d)$. Use the proper Gaussian prior ${\mathrm{\Pi }}_0=N(0,\mathrm{diag}(j^{-1-2a}{)}_{j=1}^d)$, and let ${\mathrm{\Pi }}_n(\cdot \mid Y)$ be its exact posterior under the stated forward operator and noise law. Let $m_n(Y)$ be the posterior mean and let $r_{n,\gamma }(Y)$ be the unique radius for which the Euclidean ball $C_{n,\gamma }(Y)=\{u\in {\mathbb{R}}^d:\mathrm{\parallel }u-m_n(Y){\mathrm{\parallel }}_2\le r_{n,\gamma }(Y)\}$ has posterior probability $1-\gamma$. Define the exact frequentist coverage $c_{n,\gamma }(\theta )=P_{n,\theta }\{\theta \in C_{n,\gamma }(Y)\}$ and its baseline error ${\delta }_{n,\gamma }^0={\sup }_{\theta \in {\mathrm{\Theta }}_s(R)}\mathrm{\mid }{c}_{n,\gamma }(\theta )-(1-\gamma )\mathrm{\mid }$.

For a deterministic numerical matrix ${\widehat{A}}_n\in {\mathbb{R}}^{d\times d}$ with ${\delta }_A\ge 0$ and $\mathrm{\parallel }{\widehat{A}}_n-A_p{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}\le {\delta }_A$, let ${\widehat{\mathrm{\Pi }}}_n(\cdot \mid Y)$ be the Gaussian posterior formed with ${\widehat{A}}_n$ and the same prior and noise variance. Write ${\widehat{p}}_{n,t}^Y$ for the density obtained by applying the forward variance-preserving diffusion $d{X}_t=-\frac{1}{2}\beta (t)X_{t}dt+\sqrt{\beta (t)}d{W}_t$ to $X_0\sim {\widehat{\mathrm{\Pi }}}_n(\cdot \mid Y)$, and set ${\widehat{s}}_{n,t}^Y=\mathrm{\nabla }\log {\widehat{p}}_{n,t}^Y$. A jointly Borel learned score field $s_{n,t}^Y$ is required to satisfy ${\sup }_{\theta \in {\mathrm{\Theta }}_s(R)}{E}_{Y\sim P_{n,\theta }}[\frac{1}{T}{\int }_0^T{\int }_{{\mathbb{R}}^d}\mathrm{\parallel }{s}_{n,t}^Y(x)-{\widehat{s}}_{n,t}^Y(x){\mathrm{\parallel }}_2^2{\widehat{p}}_{n,t}^Y(x)dxdt]\le {\epsilon }_s^2\mathrm{.}$ For $h=T\mathrm{/}K$, define the reverse-time Euler chain by $Z_0\sim {\widehat{p}}_{n,T}^Y$ and, for $k=0,\dots ,K-1$, $Z_{k+1}=Z_k+h[\frac{1}{2}\beta (T-kh)Z_k+\beta (T-kh)s_{n,T-kh}^Y(Z_k)]+\sqrt{h\beta (T-kh)}{\zeta }_k,$ where the ${\zeta }_k$ are independent $N(0,I_d)$ vectors independent of $Z_0$ and the observations. Draw $M$ independent copies of this chain conditionally on $Y$ and denote their terminal values by $X^{(1)},\dots ,X^{(M)}$. Let $\bar{X}=M^{-1}{\sum }_i{X}^{(i)}$, let $\widehat{r}$ be the empirical $1-\gamma$ quantile of $\{\mathrm{\parallel }{X}^{(i)}-\bar{X}{\mathrm{\parallel }}_2{\}}_{i=1}^M$ (the smallest order statistic whose empirical cumulative mass is at least $1-\gamma$), and put ${\widehat{C}}_{n,K,M}(Y)=\{u:\mathrm{\parallel }u-\bar{X}{\mathrm{\parallel }}_2\le \widehat{r}\}$. Suppose the exact credible-ball boundary satisfies the uniform margin condition ${\sup }_{\theta \in {\mathrm{\Theta }}_s(R)}{P}_{n,\theta }(\mid \mathrm{\parallel }\theta -m_n(Y){\mathrm{\parallel }}_2-r_{n,\gamma }(Y)\mid \le u)\le C_{\mathrm{a}\mathrm{c}}{u}^{\kappa }(0<u\le u_0)$ for fixed $C_{\mathrm{a}\mathrm{c}},u_0>0$ and $\kappa >0$. Determine whether a minimax-sharp nonasymptotic function $\mathrm{\Phi }$ exists uniformly over all such score fields and, if it does, find it as a function of $d,n,p,s,a,R,\sigma ,T,\gamma ,C_{\mathrm{a}\mathrm{c}},u_0,\kappa ,{\delta }_A,{\epsilon }_s,K^{-1}$, and $M^{-1\mathrm{/}2}$ such that ${\sup }_{\theta \in {\mathrm{\Theta }}_s(R)}\mid P_{n,\theta ,\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}\mathrm{r}}\{\theta \in {\widehat{C}}_{n,K,M}(Y)\}-(1-\gamma )\mid \le {\delta }_{n,\gamma }^0+\mathrm{\Phi },$ with matching lower bounds that identify when $\mathrm{\Phi }\to 0$. The probability includes observation noise and sampler randomness. The target concerns the additional coverage error introduced by the approximate forward operator, learned score, finite reverse discretization, and finite posterior sample; it does not assume that the exact Bayesian credible ball is already calibrated on every truth class.

This resolves the requested uniformly vanishing-error guarantee negatively under the original score hypothesis. Time-integrated error does not control pointwise values at a deterministic solver grid. A constant worst-case coverage bound remains possible; no impossibility of all numerical functions is asserted. No Bayesian calibration claim is imported from a conformal theorem.

For the robust minimax value ${\mathfrak{R}}_N({\rho }_X,{\rho }_A)$ in the original question reproduced below, fix $p,s>0$, $R,\sigma >0$, $N\ge 1$, ${\rho }_X\ge 0$ and $0\le {\rho }_A<1\mathrm{/}2$. Put $Q_{p,s}(R,\sigma )=R^2\min \{1,(\sigma \mathrm{/}R{)}^{4s\mathrm{/}(2s+2p+1)}\}\mathrm{.}$ There is $c_{p,s}>0$, depending only on $p,s$, such that $c_{p,s}(Q_{p,s}(R,\sigma )+R^2{\rho }_A^2)\le {\mathfrak{R}}_N({\rho }_X,{\rho }_A)\le \min \{R^2,2Q_{p,s}(R,\sigma )+R^2{\rho }_A^2\}\mathrm{.}$ These bounds hold for every training size and truth-law shift radius. For fixed positive $p,s,R$, along any sequence of the remaining parameters the risk tends to zero if and only if both $\sigma \to 0$ and ${\rho }_A\to 0$. In particular, increasing $N$ with fixed positive noise does not force the worst-law risk to zero. The original question, including both distinct shift classes, is retained verbatim under Original question.

van Maarschalkerwaart et al., full 2026 paper

Zheng et al., ICLR 2025 InverseBench paper

Bal, full inverse-problems lecture notes

Fix an integer $N\ge 1$, parameters $p,s,R,\sigma >0$, a shift radius ${\rho }_X\ge 0$, and an operator radius $0\le {\rho }_A<1\mathrm{/}2$. Let $\mathcal{H}={\mathrm{\ell }}^2(\mathbb{N})$, with norm $\mathrm{\parallel }\theta {\mathrm{\parallel }}_2^2={\sum }_{j\ge 1}{\theta }_j^2$, and define the compact truth class ${\mathrm{\Theta }}_s(R)=\{\theta \in \mathcal{H}:{\sum }_{j\ge 1}{j}^{2s}{\theta }_j^2\le R^2\},(A_p\theta {)}_j=j^{-p}{\theta }_j\mathrm{.}$ Let $\mathcal{Y}={\mathbb{R}}^{\mathbb{N}}$ have its product Borel sigma-algebra. A sequence observation with forward operator $A$ is $Y_j=(A\theta {)}_j+\sigma {\xi }_j$, where the ${\xi }_j$ are independent $N(0,1)$ variables; this defines a probability law on $\mathcal{Y}$ even though the white-noise sequence need not lie in ${\mathrm{\ell }}^2$. Write $\mathcal{P}(K)$ for the Borel probability laws on a Borel set $K$. For a Borel probability law ${\mu }_0$ on ${\mathrm{\Theta }}_s(R)$, the training data are $D_N=(({\theta }_i,Y_i){)}_{i=1}^N$ with independent ${\theta }_i\sim {\mu }_0$ and $Y_i=A_p{\theta }_i+\sigma {\xi }_i$. At test time, $\theta \sim \mu$ for another Borel probability law $\mu$ on ${\mathrm{\Theta }}_s(R)$ and the unknown forward operator is $A_{p,E}=A_p(I+E)$, where $E:{\mathrm{\ell }}^2\to {\mathrm{\ell }}^2$ is bounded linear and $\mathrm{\parallel }E{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}\le {\rho }_A$. Write $\mathcal{L}({\mathrm{\ell }}^2)$ for the bounded linear operators from ${\mathrm{\ell }}^2$ to itself. The learner receives $D_N$ and the test observation $Y$, but not $E$.

For probability laws $\mu ,{\mu }_0$ on ${\mathrm{\Theta }}_s(R)$, define their Wasserstein distance by $W_1(\mu ,{\mu }_0)={\inf }_{\mathrm{\Gamma }\in \mathrm{\Pi }(\mu ,{\mu }_0)}{\int }_{{\mathrm{\Theta }}_s(R)\times {\mathrm{\Theta }}_s(R)}\mathrm{\parallel }\theta -{\theta }_0{\mathrm{\parallel }}_2\mathrm{\Gamma }(d\theta ,d{\theta }_0),$ where $\mathrm{\Pi }(\mu ,{\mu }_0)$ is the set of couplings with those marginals. A learner is a Borel map $G_N:({\mathrm{\Theta }}_s(R)\times \mathcal{Y}{)}^N\times \mathcal{Y}\to {\mathrm{\ell }}^2$. For fixed training data its test risk is $R_{\mu ,E}(G_N\mid D_N)=E_{\theta \sim \mu ,\xi }{\parallel G_N(D_N,A_p(I+E)\theta +\sigma \xi )-\theta \parallel }_2^2,$ where $\xi$ is an independent standard Gaussian sequence. Define the robust minimax value ${\mathfrak{R}}_N({\rho }_X,{\rho }_A)={\inf }_{G_N}{\sup }_{\begin{array}{c}{\mu }_0,\mu \in \mathcal{P}({\mathrm{\Theta }}_s(R)),W_1(\mu ,{\mu }_0)\le {\rho }_X\\ E\in \mathcal{L}({\mathrm{\ell }}^2),\mathrm{\parallel }E{\mathrm{\parallel }}_{\mathrm{o}\mathrm{p}}\le {\rho }_A\end{array}}{E}_{D_N\sim ({\mu }_0,A_p)}{R}_{\mu ,E}(G_N\mid D_N)\mathrm{.}$ Determine matching upper and lower bounds for ${\mathfrak{R}}_N({\rho }_X,{\rho }_A)$ as a function of $N,{\rho }_X,{\rho }_A,p,s,R,\sigma$, and identify necessary and sufficient joint regimes in which it tends to zero. In particular, the bound must distinguish source-to-test truth-law shift from perturbation of the conditional measurement law induced by $E$; no access to $E$ is available at deployment.