Score and diffusion methods for inverse data — Examples and Counterexamples

Choose $N=1$, $t_0=0$, $t_1=1$, state space $\mathbb{R}$, and observation space $(\mathsf{Y},\mathcal{Y})=(\{\ast \},2^{\{\ast \}})$ with $y=\ast$. Take the unique forward map $\mathcal{A}:\mathbb{R}\to \mathsf{Y}$, likelihood ${\mathrm{\ell }}_y\equiv 1$ relative to counting measure, terminal law ${\nu }_1={\delta }_0$ with deterministic initialization at $0$, zero score field, a singleton correction space with constant correction, the Borel constant kernel $Q_m(B\mid x,y,a)={\mu }_m(B)$, and no projection. The kernel is a probability kernel because ${\mu }_m$ has total mass one. If ${\xi }_1,\dots ,{\xi }_m$ are independent fair coin flips, the sampler that returns $1$ exactly when every ${\xi }_i=1$ and returns $0$ otherwise has law ${\mu }_m$ and uses at most $m$ flips. Thus [F2] permits the one-step instance and its returned state has law ${\mu }_m$.

For every bounded continuous $f:\mathbb{R}\to \mathbb{R}$, ${\displaystyle \int fd{\mu }_m-\int fd\pi =2^{-m}(f(1)-f(0))⟶0}$. Thus the defining test integrals converge, so ${\mu }_m\Rightarrow \pi$ by [F1].

For each finite integer $m\ge 1$, ${\mu }_m(\{1\})=2^{-m}>0=\pi (\{1\})$, hence ${\mu }_m\mathrm{\ne }\pi$. In particular this holds at the first index $m=1$ as well as every later finite index.

The output laws therefore converge weakly to the target but are not equal to it at any finite index, so this convergence statement supplies no finite-index exactness claim.

In the explicit-score family, [F1] identifies the exact conditional score. Under [A2], the initial sampler state has law ${\mathrm{\Pi }}_{y,N}={\nu }_N$. If its state at step $k$ has law ${\mathrm{\Pi }}_{y,k}$, applying $R_k$ gives $B$ probability $\int R_k(B\mid x,y){\mathrm{\Pi }}_{y,k}(dx)={\mathrm{\Pi }}_{y,k-1}(B)$. Induction down to $k=1$ therefore gives output law ${\mathrm{\Pi }}_{y,0}={\pi }_y$. This applies to [F2] with these kernels and no projection. Almost-sure success makes completed status have probability one, introducing no conditioning bias. The sampled joint path need not have law ${\mathrm{\Pi }}_y$; only its one-time marginals are used.

Since $q\ll {\pi }_y$ and both are probability measures, the Radon–Nikodym density $f=dq\mathrm{/}d{\pi }_y$ has integral one. Convexity of $t\mapsto t\log t$ and Jensen give $D_{\mathrm{K}\mathrm{L}}(q\mathrm{\parallel }{\pi }_y)=\int f\log fd{\pi }_y\ge 0$, with equality only when $f=1$ ${\pi }_y$-almost everywhere. Since ${\pi }_y(x)=Z_y^{-1}{\mathrm{\ell }}_y(x){\rho }_0(x)$, the integrability in [A3] gives $\mathrm{ELBO}(q)=\int q\log (Z_y{\pi }_y\mathrm{/}q)d\lambda =\log Z_y-\int q\log (q\mathrm{/}{\pi }_y)d\lambda \le \log Z_y$, with equality exactly when $q={\pi }_y$. Thus the variational output law is its selected $q$; it is the target law in the exact case $q={\pi }_y$, and it differs from the target precisely when the induced probability measures differ, equivalently when the densities disagree on a set of positive $\lambda$-measure.

Under [A4], repeated execution of the deterministic latent optimizer returns the single value $G(\widehat{z})$, so its output law is ${\delta }_{G(\widehat{z})}$. If ${\pi }_y$ is not a point mass, this point mass is not ${\pi }_y$. The stated objective contains only the measurement residual; it does not specify the negative log posterior or a global optimizer of that posterior objective, so the given inputs establish no MAP conclusion.

If a Markov chain starts with law ${\pi }_y$, one transition has law ${\pi }_y{K}_y={\pi }_y$ by [F3]; induction gives the same marginal after every finite number of transitions. In contrast, on the Borel space $\mathbb{R}$ let $\pi =({\delta }_0+{\delta }_1)\mathrm{/}2$ and let $K(x,B)={\mathbf{1}}_B(x)$. Then $\pi K=\pi$, but a chain starting at ${\delta }_0$ remains ${\delta }_0$ after each finite step and is never $\pi$.

To check the failure boundary, take $d=p=1$, the Borel observation space $\mathsf{Y}=\mathbb{R}$, $y=0$, and $\mathcal{A}(x)=0$. Let ${\mu }_0={\pi }_y=({\delta }_0+{\delta }_1)\mathrm{/}2$, dominated by $\lambda ={\delta }_0+{\delta }_1$ with ${\rho }_0\equiv 1\mathrm{/}2$. Take $Y\equiv 0$, whose conditional density is identically one relative to the finite observation measure ${\delta }_0$ on $\mathbb{R}$; thus ${\mathrm{\ell }}_y\equiv 1$ and $Z_y=1$. Use $N=2$, $t_k=k$, terminal law ${\nu }_2={\delta }_0$, zero score, singleton correction spaces ${\mathsf{C}}_1={\mathsf{C}}_2=\{\ast \}$, corrections, for every $v\in \mathbb{R}$, $C_2(x,v)=\ast$ and $C_1(x,v)=\mathrm{\perp }$ exactly when $x=1$ (otherwise $\ast$), kernels $Q_{\theta ,2}(B\mid x,v,\ast )={\pi }_y(B)$ and $Q_{\theta ,1}(B\mid x,v,\ast )={\delta }_x(B)$, and no projection. Initialize deterministically at zero, sample $Q_{\theta ,2}$ with one fair coin, and sample $Q_{\theta ,1}$ by returning the current state; these operations have finite budgets. The exact path law is ${\mathrm{\Pi }}_y(d{x}_0,d{x}_1,d{x}_2)={\delta }_0(d{x}_2){\pi }_y(d{x}_1){\delta }_{x_1}(d{x}_0)$, so its reverse kernels draw $X_1$ from ${\pi }_y$ and then set $X_0=X_1$. Completion occurs exactly when $X_1=0$, so its probability is $1\mathrm{/}2$ and its conditional output law is ${\delta }_0$, not ${\pi }_y$. Thus allowing the failure branches in [F2] invalidates an unconditional exactness inference from the reverse kernels alone.

A realized weighted empirical particle measure ${\widehat{\pi }}_K={\sum }_{j=1}^K{w}_j{\delta }_{x_j}$ is supported on the finite set $S=\{x_1,\dots ,x_K\}$. By [A5] and finite additivity, ${\pi }_y(S)\le {\sum }_{j=1}^K{\pi }_y(\{x_j\})=0$, while ${\widehat{\pi }}_K(S)=1$, so the finite empirical measure is not the target. This does not determine the marginal law of a randomly selected particle; that exactness is a separate algorithm-specific claim.