Conditional flow matching for inverse posteriors

Let $a\le b$, let $A\ge 0$, let $u:[a,b]\to [0,\mathrm{\infty })$ be continuous, and let $\mathrm{\ell }:[a,b]\to [0,\mathrm{\infty })$ be Lebesgue measurable with ${\int }_a^b\mathrm{\ell }(s)ds<\mathrm{\infty }$. If $u(t)\le A+{\int }_a^t\mathrm{\ell }(s)u(s)ds(a\le t\le b),$ then $u(t)\le A\exp \text{\hspace{0.17em}⁣}({\int }_a^t\mathrm{\ell }(s)ds)(a\le t\le b)\mathrm{.}$ If instead $u(t)\le A+{\int }_t^b\mathrm{\ell }(s)u(s)ds$ for every $t\in [a,b]$, then $u(t)\le A\exp \text{\hspace{0.17em}⁣}({\int }_t^b\mathrm{\ell }(s)ds)(a\le t\le b)\mathrm{.}$

Teschl, §2.4, Lemma 2.7

CMU ODE notes, Grönwall inequality

Assume Countable Choice and Dependent Choice. Let $T>0$, let $d,m\ge 1$, and let $b:[0,T]\times {\mathbb{R}}^d\times {\mathbb{R}}^m\to {\mathbb{R}}^d$ be jointly continuous on the product Euclidean domain. Suppose that for every compact $K\subseteq {\mathbb{R}}^m$ there are nonnegative Lebesgue-measurable $a_K,L_K\in L^1([0,T])$ such that for almost every $t$, simultaneously for all $y\in K$ and $x,x^{\mathrm{\prime }}\in {\mathbb{R}}^d$, $\mathrm{\mid }b(t,0,y)\mathrm{\mid }\le a_K(t),\mathrm{\mid }b(t,x,y)-b(t,x^{\mathrm{\prime }},y)\mathrm{\mid }\le L_K(t)\mathrm{\mid }x-x^{\mathrm{\prime }}\mathrm{\mid }\mathrm{.}$ Here $L^1([0,T])$ means the functions have finite Lebesgue integral of their absolute values. Then for every $x_0\in {\mathbb{R}}^d$ and $y\in {\mathbb{R}}^m$ there is a unique continuously differentiable solution $x_y(\cdot ;x_0)$ of $\dot{x}(t)=b(t,x(t),y),x(0)=x_0,$ on all of $[0,T]$. Its terminal map $T_y(x_0):=x_y(T;x_0)$ is a bijection with jointly continuous forward and inverse maps in $(x_0,y)$, and for all $x_0,x_1$ and $y\in K$, $\mathrm{\mid }{T}_y(x_0)-T_y(x_1)\mathrm{\mid }\le e^{{\int }_0^T{L}_K(t)dt}\mathrm{\mid }{x}_0-x_1\mathrm{\mid },\mathrm{\mid }{T}_y^{-1}(x_0)-T_y^{-1}(x_1)\mathrm{\mid }\le e^{{\int }_0^T{L}_K(t)dt}\mathrm{\mid }{x}_0-x_1\mathrm{\mid }\mathrm{.}$

In the separate C${}^1$ clause, suppose additionally that the state derivative $D_{x}b$ exists and is jointly continuous. Then for fixed $y$, $T_y$ is a bijection and both $T_y$ and its inverse are continuously differentiable. For a measurable map $S$ and a measure $\mu$, define its image measure by $S_{\mathrm{\#}}\mu (E):=\mu (S^{-1}(E))$. The derivative $A_y(t,x_0):=D_{x_0}{x}_y(t;x_0)$ solves ${\dot{A}}_y(t,x_0)=D_{x}b(t,x_y(t;x_0),y)A_y(t,x_0),A_y(0,x_0)=I_d,$ with Liouville formula $\det A_y(t,x_0)=\exp \text{\hspace{0.17em}⁣}({\int }_0^t\mathrm{tr}{D}_{x}b(s,x_y(s;x_0),y)ds)\mathrm{.}$ Consequently, if a probability measure $\eta$ on ${\mathbb{R}}^d$ has Lebesgue density $q$, meaning $\eta (E)={\int }_{E}q(x)dx$ for Borel $E$, the image measure $T_y{}_{\mathrm{\#}}\eta$ has density $z⟼q(T_y^{-1}(z))\mathrm{\mid }\det D{T}_y^{-1}(z)\mathrm{\mid }\mathrm{.}$ The global Lipschitz clause alone gives a bi-Lipschitz flow and does not assert a Jacobian density formula.

Teschl, §§2.3–2.6, full ODE text

CMU ODE notes, existence, flow, and Liouville formula

Folland, Theorem 2.47, change of variables

Assume the Axiom of Choice. Let $\pi$ be a probability measure on ${\mathbb{R}}^d\times {\mathbb{R}}^m$, where $d,m\ge 1$, and let ${\pi }_Y(B):=\pi ({\mathbb{R}}^d\times B)$ be its second marginal. Let $\eta$ be a probability measure on ${\mathbb{R}}^d$, and let $y\mapsto {\pi }_y$ be a probability kernel satisfying $\pi (A\times B)={\int }_B{\pi }_y(A){\pi }_Y(dy)$ for all Borel $A\subseteq {\mathbb{R}}^d$ and $B\subseteq {\mathbb{R}}^m$. Suppose $T:{\mathbb{R}}^d\times {\mathbb{R}}^m\to {\mathbb{R}}^d\times {\mathbb{R}}^m$ is Borel and triangular, $T(x,y)=(T_y(x),y)$, with each $T_y$ a Borel bijection, and define the image measure by $T_{\mathrm{\#}}\mu (E):=\mu (T^{-1}(E))$. If $T_{\mathrm{\#}}(\eta \otimes {\pi }_Y)=\pi ,$ then $T_y{}_{\mathrm{\#}}\eta ={\pi }_y$ as probability measures for ${\pi }_Y$-almost every $y$. By disintegration on standard Borel spaces, such a kernel ${\pi }_y$ exists and is unique outside one ${\pi }_Y$-null set. In a proper Bayesian model, when $\pi$ is the joint parameter/data law, this conditional kernel is the posterior kernel; the conclusion says an exact triangular transport maps the reference law to the data-conditioned posterior for almost every datum.

For the triangular terminal flow in the preceding ODE lemma, joint continuity supplies the Borel map and each fiber is a bijection. The theorem requires only the exact joint pushforward identity; it does not require a density or a Jacobian formula.

The ODE flow result supplies a jointly continuous triangular terminal map with bijective fibers.

The Bayesian model definition identifies the conditional kernel of the proper joint law as a posterior version.

Assume the Axiom of Choice. Let $d,m,n,B\ge 1$, let $\eta$ be a reference probability on ${\mathbb{R}}^d$, and let $\pi$ be the proper joint law of an unknown $X_1\in {\mathbb{R}}^d$ and data $Y\in {\mathbb{R}}^m$ in a specified Bayesian inverse model. Write ${\pi }_Y$ for the data marginal and ${\pi }_y$ for a posterior-kernel version of the conditional law of $X_1$ given $Y=y$. Assume ${\int }_{{\mathbb{R}}^d\times {\mathbb{R}}^m}\mathrm{\mid }{x}_1{\mathrm{\mid }}^2\pi (d{x}_1,dy)+{\int }_{{\mathbb{R}}^d}\mathrm{\mid }{x}_0{\mathrm{\mid }}^2\eta (d{x}_0)<\mathrm{\infty }\mathrm{.}$ Choose a jointly measurable parameterized vector-field family $v_{\theta }:[0,1]\times {\mathbb{R}}^d\times {\mathbb{R}}^m\to {\mathbb{R}}^d$, with parameter set $\mathrm{\Theta }\subseteq {\mathbb{R}}^p$ for some $p\ge 1$, whose candidates have finite population squared loss under the training law below.

Declared inputs. Supply independent sampling routines for $\mathrm{Unif}[0,1]$, $\eta$, and $\pi$; here $\mathrm{Unif}[0,1]$ is Lebesgue probability on the unit interval. Supply an initialiser $\mathrm{I}\mathrm{n}\mathrm{i}\mathrm{t}$ returning a parameter in $\mathrm{\Theta }$ or failure; an optimizer update callable $\mathrm{U}\mathrm{p}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e}(k,\theta ,{\widehat{\mathcal{L}}}_n,{\mathcal{D}}_n)$ returning a parameter in $\mathrm{\Theta }$ or failure, with each call counting as one optimizer step; a named stopping statistic callable $S(k,\theta ,{\widehat{\mathcal{L}}}_n,{\mathcal{D}}_n)$ returning a finite real value when evaluation succeeds; a finite step budget $M\ge 1$ and tolerance ${\epsilon }_{\mathrm{o}\mathrm{p}\mathrm{t}}>0$; a flow-check callable; and a numerical ODE solver with a declared finite step or tolerance budget $h$. Every callable must return its specified value or a failure status in finite time. Any randomization used by these callables is part of the declared input.

Training and optimizer instructions. Draw $n$ independent triples $(t_i,X_{0i},(X_{1i},Y_i))\sim \mathrm{Unif}[0,1]\otimes \eta \otimes \pi ,$ and form $X_{t_i}=(1-t_i)X_{0i}+t_i{X}_{1i}$ and $V_i=X_{1i}-X_{0i}$. Let ${\mathcal{D}}_n=((t_i,X_{t_i},Y_i,V_i){)}_{i=1}^n$ and evaluate the empirical squared loss ${\widehat{\mathcal{L}}}_n(\theta )=\frac{1}{n}{\sum }_{i=1}^n\mid v_{\theta }(t_i,X_{t_i},Y_i)-V_i{\mid }^2\mathrm{.}$ If sampling fails, return sampling-failed. Otherwise call $\mathrm{I}\mathrm{n}\mathrm{i}\mathrm{t}$; if it fails or returns a parameter outside $\mathrm{\Theta }$, return optimizer-failed with no fitted field. Set $k=0$ and repeat the following steps in order: evaluate ${\widehat{\mathcal{L}}}_n({\theta }_k)$ and $S(k,{\theta }_k,{\widehat{\mathcal{L}}}_n,{\mathcal{D}}_n)$; if either evaluation fails or either value is nonfinite, return optimizer-failed with no fitted field; if $S\le {\epsilon }_{\mathrm{o}\mathrm{p}\mathrm{t}}$, stop with status criterion-met; if $k=M$, stop with status budget-exhausted; otherwise call $\mathrm{U}\mathrm{p}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e}(k,{\theta }_k,{\widehat{\mathcal{L}}}_n,{\mathcal{D}}_n)$, returning optimizer-failed with no fitted field if the call fails, returns a parameter outside $\mathrm{\Theta }$, or yields a nonfinite value, and then set ${\theta }_{k+1}$ to its output and increase $k$ by one. For either successful stopping status, return $\widehat{v}=v_{{\theta }_k}$, its achieved training loss, the last value of $S$, $k$, $M$, ${\epsilon }_{\mathrm{o}\mathrm{p}\mathrm{t}}$ and the optimizer status. A finite-data stopping status does not certify $\widehat{v}=v^{\ast }$.

The population least-squares target is the conditional-expectation class $v^{\ast }(t,x,y)=\mathbb{E}[X_1-X_0\mid \sigma (t,X_t,Y)]$ coordinatewise, evaluated at $(t,x,y)$ through a chosen measurable representative. A conditional expectation is only an almost-sure equivalence class, so this formula does not specify an off-support pointwise extension.

Flow and sampling instructions. For a query datum $y$, call the flow checker on $\widehat{v}$ and the data-parameter domain containing $y$. It returns pass only when it certifies all hypotheses of the ODE flow lemma, including joint continuity and the integrable global spatial-Lipschitz and origin-growth bounds on compact data-parameter sets containing $y$. If it does not return pass, return field-regularity-failed with the optimizer record and no terminal samples. Otherwise, for $j=1,\dots ,B$, draw independent $Z_0^j\sim \eta$ and call the declared solver to integrate on $0\le t\le 1$ ${\dot{Z}}_t^j=\widehat{v}(t,Z_t^j,y),Z_0^j,$ with budget $h$. Record the returned finite endpoint as the terminal approximation ${\widehat{Z}}_1^j$ only when the solver reports success; treat a success response with a nonfinite endpoint as a failed integration. Also record each status and any certified solver error estimate, marking an unavailable estimate unquantified. Continue through all $B$ calls after a solver failure. Return the successful terminal approximations and all per-call records, together with the field and optimizer records; the overall status is completed if every integration succeeds and solver-failed otherwise. A sampling-call failure returns sampling-failed and the records collected so far.

If the exact triangular flow of $\widehat{v}$ were known to satisfy $T_{\mathrm{\#}}(\eta \otimes {\pi }_Y)=\pi$, the exact transport theorem would imply $T_y{}_{\mathrm{\#}}\eta ={\pi }_y$ for ${\pi }_Y$-almost every $y$. Finite regression loss, optimizer termination, and solver success alone do not establish this joint pushforward identity or exact posterior sampling. Record posterior model, discretization, kernel, burn-in, Monte Carlo and solver components under the declared posterior-computation error budget when applicable. The budget does not supply a learned-flow regression or optimizer-error estimate; report those separately as measured or certified quantities, and mark unavailable quantities unquantified rather than zero.

A proper prior and likelihood model induce a joint prior-predictive law whose data-indexed conditional kernel is the posterior version.

Conditional expectation is the measurable integrable representative characterized by its integrals on the conditioning sigma-algebra.

Under AC, conditional expectation is the unique almost-sure least-squares minimizer in the conditioning $L^2$ space.

On standard Borel conditioning coordinates, a conditional expectation is represented by a measurable function of those coordinates.

The integrable global Lipschitz and growth hypotheses give a unique global ODE flow and quantitative bi-Lipschitz bounds.

An exact joint triangular pushforward gives the conditional posterior pushforward almost everywhere.

The posterior error ledger has six adjacent model, solver, discretization, invariant-kernel, burn-in and Monte Carlo terms, but no learned-flow training term.

Blechschmidt et al., §§3.1–3.2 and Appendix B

Tong et al., Theorem 3.2 and §3.2

Zhai et al., §§3.1–3.2

Assume the Axiom of Choice. Let $d,m,n\ge 1$, let $\eta$ be a probability measure on ${\mathbb{R}}^d$, and let $\pi$ be a proper Bayesian joint law on ${\mathbb{R}}^d\times {\mathbb{R}}^m$ with data marginal ${\pi }_Y$ and posterior-kernel version ${\pi }_y$. Assume $\int \mathrm{\mid }{x}_0{\mathrm{\mid }}^2\eta (d{x}_0)+\int \mathrm{\mid }{x}_1{\mathrm{\mid }}^2\pi (d{x}_1,dy)<\mathrm{\infty }$. Let $t$, $X_0$ and $(X_1,Y)$ be independent with law $\mathrm{Unif}[0,1]\otimes \eta \otimes \pi$, where $X_t=(1-t)X_0+t{X}_1$, and $V=X_1-X_0$ has finite second moment. Write $\mathcal{L}(v):=\mathbb{E}\mathrm{\mid }v(t,X_t,Y)-V{\mathrm{\mid }}^2$ and let ${\widehat{\mathcal{L}}}_n(v)$ be the empirical mean of this squared residual over $n\ge 1$ independent training triples. Then the conditional-expectation class $v^{\ast }(t,X_t,Y)=\mathbb{E}[V\mid \sigma (t,X_t,Y)]$ admits a measurable representative in $(t,x,y)$ and uniquely minimizes $\mathcal{L}$ over square-integrable predictors measurable in these coordinates, up to almost-sure equality. For every fixed measurable candidate $v$ with finite population loss, $\mathbb{E}[{\widehat{\mathcal{L}}}_n(v)]=\mathcal{L}(v)\mathrm{.}$ This expectation identity is for a fixed candidate, not a data-dependent fitted field.

If a fitted field satisfies all hypotheses of the global ODE flow lemma, it has an exact triangular terminal map $T(x,y)=(T_y(x),y)$. If also $T_{\mathrm{\#}}(\eta \otimes {\pi }_Y)=\pi$, then $T_y{}_{\mathrm{\#}}\eta ={\pi }_y$ as probability measures outside one ${\pi }_Y$-null set. These conclusions make no assertion that finite training or numerical solver success establishes the exact joint pushforward identity.

Conditional flow matching supplies the training law and empirical objective.

Blechschmidt et al., section 3.2 and Theorem 3.1

Assume the Axiom of Choice. Its implications supply Dependent Choice and Countable Choice for the finite-dimensional ODE flow assumptions.

Let $\mathcal{H}$ be a separable real Hilbert space, let $K\subseteq {\mathbb{R}}^m$ be compact, and let $y\mapsto {\mathrm{\Pi }}^y$ be the pointwise potential posterior of a proper Bayesian inverse model, with positive finite evidence and finite second moment for every $y\in K$. Let $P_n$ be finite-rank orthogonal projections on $\mathcal{H}$ with $P_{n}x\to x$ for every $x\in \mathcal{H}$, let ${\mathcal{H}}_n=P_n\mathcal{H}$, and let ${\mathrm{\Pi }}_n^y$ be proper posteriors defined by declared finite-rank forward/potential approximations on ${\mathcal{H}}_n$, with finite second moments for every $y\in K$. Write $i_n:{\mathcal{H}}_n\hookrightarrow \mathcal{H}$ for inclusion and define the posterior model error $e_{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t},n}:={\sup }_{y\in K}{W}_{2,\mathcal{H}}(i_{n\mathrm{\#}}{\mathrm{\Pi }}_n^y,{\mathrm{\Pi }}^y),$ where $W_{2,\mathcal{H}}(\mu ,\nu )$ is the infimum, over joint laws with marginals $\mu ,\nu$, of the square root of the expected squared Hilbert distance. This term includes both posterior-potential and finite-rank model error; the known Hellinger potential/discretization estimates would need an explicit moment or tail argument to control it in $W_2$.

Let $\pi$ be the prior-predictive joint law of $(X_1,Y)$ and assume the global prior-predictive moment condition ${\int }_{\mathcal{H}\times {\mathbb{R}}^m}\mathrm{\parallel }{x}_1{\mathrm{\parallel }}_{\mathcal{H}}^2\pi (d{x}_1,dy)<\mathrm{\infty }\mathrm{.}$ Let $\eta$ be an independent reference probability on $\mathcal{H}$ with finite second moment, and use the product training law $\mathrm{Unif}[0,1]\otimes \eta \otimes \pi$. Let $X_t=(1-t)X_0+t{X}_1$ and let $v^{\ast }(t,X_t,Y)=\mathbb{E}[X_1-X_0\mid \sigma (t,X_t,Y)]$ be a chosen continuum conditional regression version. For projected endpoints, define $v_n^{\ast }(t,P_n{X}_t,Y)=\mathbb{E}[P_n(X_1-X_0)\mid \sigma (t,P_n{X}_t,Y)]$ and the finite-rank conditional-target discrepancy $e_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t},n}:={\sup }_{y\in K}{\left({\int }_0^1\mathbb{E}\text{\hspace{0.17em}⁣}[{\parallel P_n{v}^{\ast }(t,X_t,y)-v_n^{\ast }(t,P_n{X}_t,y)\parallel }_{\mathcal{H}}^2\mid Y=y]dt\right)}^{1\mathrm{/}2}\mathrm{.}$ The conditional expectation versions at $Y=y$ must be chosen on all $y\in K$ for these suprema. For a measurable vector field $v:[0,1]\times {\mathcal{H}}_n\times K\to {\mathcal{H}}_n$, define its pointwise conditional population loss by ${\mathcal{L}}_{n,y}(v):={\int }_0^1\mathbb{E}\text{\hspace{0.17em}⁣}[{\parallel v(t,P_n{X}_t,y)-P_n(X_1-X_0)\parallel }_{\mathcal{H}}^2\mid Y=y]dt,$ using the selected conditional-law version. For a finite training sample of size $N$, let ${\bar{v}}_{n,N}$ be an empirical-risk minimizer in the specified vector-field class, when it exists. On realizations where the declared optimizer returns a field (status criterion-met or budget-exhausted), let ${\widehat{v}}_{n,N}$ be that field; the optimizer-failed case has no learned field and is outside this formulation. Define the uniform conditional statistical and optimizer excess-risk terms by $e_{\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{t},n,N}^2:={\sup }_{y\in K}[{\mathcal{L}}_{n,y}({\bar{v}}_{n,N})-{\inf }_{v\in {\mathcal{V}}_n}{\mathcal{L}}_{n,y}(v)],e_{\mathrm{o}\mathrm{p}\mathrm{t},n,N}^2:={\sup }_{y\in K}{[{\mathcal{L}}_{n,y}({\widehat{v}}_{n,N})-{\mathcal{L}}_{n,y}({\bar{v}}_{n,N})]}_{+}$ where ${\mathcal{V}}_n$ is the chosen class and ${\mathcal{L}}_{n,y}$ is its population conditional squared regression loss at datum $y$. Let $e_{\mathrm{s}\mathrm{o}\mathrm{l}\mathrm{v}\mathrm{e}\mathrm{r},n,N,h}$ be a certified $W_2$ coupling bound between the numerical endpoint law at solver budget $h$ and the exact learned-flow endpoint law. Finally, let $\mathrm{\Lambda }$ be a proposed bound, uniform in $n$ and $y\in K$, for the time-integrated spatial Lipschitz constants of the learned drifts.