Benchmarks, prior shift, and solver admissibility

Fix measurable spaces $(X,\mathcal{X})$ and $(Y,\mathcal{Y})$, a metric target space $(Q,d_Q)$ with its Borel sigma-algebra $\mathcal{Q}$, and a nonempty truth class $U\subseteq X$ with inherited sigma-algebra $\mathcal{U}=\{A\cap U:A\in \mathcal{X}\}$. Let $q:U\to Q$ and $F:U\to Y$ be measurable target and forward maps. The admissible state set is a declared nonempty $\mathcal{U}$-measurable subset $C\subseteq U$. The observation model is a specified probability kernel $K(dy\mid x)$ from $(U,\mathcal{U})$ to $(Y,\mathcal{Y})$; in an additive-noise model, when $Y$ has a measurable addition operation, it is the law of $F(x)+\epsilon$ for a declared noise law of $\epsilon$, while nonadditive or model-dependent observation noise is encoded directly by $K$.

The forward solver, when a numerical evaluation is used, is the specified procedure for evaluating this model, including its discretization, tolerances, finite call budget, and any model-specific admissibility test; exact evaluation is the special case with no numerical-solver error. A failed admissibility test or solver call is recorded with its failure status and is not silently omitted from the benchmark results.

A benchmark specification also declares a training law $P_{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}}$ on a named measurable space for all random variables used to fit a method or select its hyperparameters; those variables may be paired or unpaired. It declares a test law $P_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ on $(Y\times Q,\mathcal{Y}\otimes \mathcal{Q})$ for evaluation pairs $(Y,q(X))$, and any separate validation law used for tuning. The training and test laws need not agree. Each candidate method is identified by its fitting procedure and by its deployed measurable deterministic reconstruction rule $g:Y\to Q$ or randomized reconstruction kernel $L(da\mid y)$ from $Y$ to $Q$. Its population test risk is the risk under $P_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ as defined by the inverse-risk functional; a reported finite-sample score must additionally name its test sample and empirical aggregation rule. No risk transfer or generalization bound follows from the benchmark specification.

Before comparison, the specification fixes a nonnegative product-measurable loss $\mathrm{\ell }:Q\times Q\to [0,+\mathrm{\infty }]$, an uncertainty diagnostic, and a finite compute-budget vector. An uncertainty diagnostic is a declared measurable statistic of the method outputs and, when applicable, target labels, together with its target quantity; for example, if a reported set-valued estimate $C(y)\subseteq Q$ has a product-measurable membership event $\{(y,a):a\in C(y)\}\in \mathcal{Y}\otimes \mathcal{Q}$, its test coverage is $P_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}(\{(y,a)\in Y\times Q:a\in C(y)\})$. The budget vector records the applicable maximum counts of forward-model queries, adjoint queries, gradient evaluations, ODE right-hand-side evaluations, solver or sampling steps, and elapsed runtime; inapplicable counts are marked as not applicable, not zero. It also records the hardware, numerical precision, software/model versions, random seeds, tuning data, and stopping tolerances needed to interpret those counts. Each method's result records its metric values, uncertainty diagnostic, realized compute counts, solver status, and failed runs. A comparison is tied to the declared test law, test sample, loss, solver, admissibility rules, and tuning protocol; when budgets or other conditions differ, those differences are reported as part of the comparison. The specification defines an evaluation protocol, not a theorem ranking algorithms.

The arrows label declared information flow into fitting, observation generation, and evaluation. The diagram asserts no equality between $P_{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}}$ and $P_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}$ and no risk-transfer result.

Risk under a specified joint evaluation law

Point reconstruction rules and randomized posterior kernels

Separate forward operators, observation laws, and prior support

A model-specific finite solver/admissibility contract

Bal, Introduction to Inverse Problems, §1.1 and §§8.2–8.3, full lecture notes

Zheng et al., InverseBench, ICLR 2025, full paper and appendices

Zheng et al.'s InverseBench reports 14 plug-and-play diffusion-prior algorithms and domain-specific baselines on five selected cases: first-Born optical scattering, raw multi-coil knee MRI, black-hole very-long-baseline interferometry, full waveform inversion, and two-dimensional vorticity-form Navier–Stokes inversion. For these experiments, the authors report that diffusion-prior methods often outperform conventional baselines when the training data are suitable, while a classical full-waveform-inversion optimizer initialized from a smoothed ground truth outperforms the tested diffusion-prior methods. In cases with closed-form forward models, methods using more forward-gradient queries can be more accurate and more expensive; the paper reports that this trend fails for its numerical PDE cases. Its Navier–Stokes split has one validation case and ten test cases. The paper reports step-size/Courant–Friedrichs–Lewy (CFL) failures in its PDE experiments. Its Figure 5 out-of-distribution diagnostics feed digit images to a black-hole prior trained on simulated black-hole images, and nine-cell scattering sources to a prior trained on one-to-six-cell sources; for the black-hole case, the authors report outputs with the prior's characteristic ring feature. These are empirical observations for the named data, solvers, methods, tuning protocol, and hardware; they establish no universal ranking, stability bound, posterior exactness, risk-transfer bound, or robustness theorem.

There is a first-Born notation discrepancy between the paper and its released implementation. Appendix B.1 defines a contrast-dependent total field and then displays a sensor equation with that field multiplied by the contrast, which does not itself exhibit a fixed linear map. The pinned implementation instead passes the fixed incident field into the sensor-propagation routine and constructs its cached linear matrix from that fixed field. This page therefore identifies the implemented benchmark as the first-Born linear case and does not infer linearity from the printed contrast-dependent expression.

Benchmark specification and reporting protocol

Risk under a named joint evaluation law

Learned estimators, score fields, and posterior samplers carry no automatic guarantee

Operator and prior mismatch are distinct and imply no automatic robustness bound

Solver admissibility and CFL thresholds are model-specific

Exact diffusion scores versus learned or plug-in approximations

Zheng et al., InverseBench, ICLR 2025, full paper and appendices

InverseBench implementation, pinned inverse_scatter.py source