Finite Gaussian forward chain and Gaussian reverse model

Assume AC. Let $d,T\ge 1$ be finite integers, let $I_d$ be the $d\times d$ identity matrix, and let ${\beta }_1,\dots ,{\beta }_T\in (0,1)$. Set ${\alpha }_t=1-{\beta }_t$, ${\bar{\alpha }}_0=1$, and ${\bar{\alpha }}_t={\prod }_{s=1}^t{\alpha }_s$. Write ${\lambda }_d$ for Lebesgue measure on ${\mathbb{R}}^d$ and define, for $v>0$, ${\varphi }_{d,v}(y;m)=(2\pi v{)}^{-d\mathrm{/}2}\exp \text{\hspace{0.17em}⁣}(-\frac{\mathrm{\parallel }y-m{\mathrm{\parallel }}_2^2}{2v}),y,m\in {\mathbb{R}}^d\mathrm{.}$ The notation $\mathrm{\parallel }z{\mathrm{\parallel }}_2^2={\sum }_{i=1}^d{z}_i^2$ is the squared Euclidean norm. The displayed function is the Lebesgue density of $N_d(m,v{I}_d)$.

For each fixed $x_0\in {\mathbb{R}}^d$, the forward chain has path density on $({\mathbb{R}}^d{)}^T$ $q(x_{1:T}\mid x_0)={\prod }_{t=1}^T{\varphi }_{d,{\beta }_t}(x_t;\sqrt{{\alpha }_t}{x}_{t-1})\mathrm{.}$ Thus its transition from $x_{t-1}$ to $x_t$ is Gaussian with mean $\sqrt{{\alpha }_t}{x}_{t-1}$ and covariance ${\beta }_t{I}_d$. Each factor is a normalized density, and finite successive integration of the nonnegative product gives a probability density.

For a generic adjacent pair, the two model directions are shown below. The arrows indicate the direction in which each conditional density generates its target state; they assert no equality or inverse relation between the kernels.

diagram failed to render:
\begin{tikzpicture}[>=stealth]
  \node (previous) at (0,0) {$x_{t-1}$};
  \node (current) at (4,0) {$x_t$};
  \draw[->] (previous) to[bend left=24] node[midway,above] {$q(x_t\mid x_{t-1})$} (current);
  \draw[->] (current) to[bend left=24] node[midway,below] {$p_{\theta,t}(x_{t-1}\mid x_t)$} (previous);
\end{tikzpicture}

A learned reverse Gaussian model consists of finite-valued Borel maps ${\mu }_{\theta ,t}:{\mathbb{R}}^d\to {\mathbb{R}}^d$ and fixed variances ${\sigma }_t^2\in (0,\mathrm{\infty })$, for $1\le t\le T$. It is the joint density $p_{\theta }(x_{0:T})={\varphi }_{d,1}(x_T;0){\prod }_{t=1}^T{\varphi }_{d,{\sigma }_t^2}(x_{t-1};{\mu }_{\theta ,t}(x_t))\mathrm{.}$ Its prior is $p(x_T)=N_d(0,I_d)$, and its marginal density at $x_0$ is $p_{\theta }(x_0)={\int }_{({\mathbb{R}}^d{)}^T}{p}_{\theta }(x_{0:T})d{\lambda }_d(x_1)\cdots d{\lambda }_d(x_T),$ whenever this integral is finite. The joint reverse density is normalized by successive integration of its Gaussian transition factors followed by the prior density.

For $t=1$, the forward-chain conditional $q(x_0\mid x_1,x_0)$ assigns probability one to every Borel set containing $x_0$ and zero to every Borel set not containing $x_0$; it is not a Lebesgue density. The separate learned reverse likelihood $p_{\theta }(x_0\mid x_1)$ is the Gaussian density above with variance ${\sigma }_1^2>0$.

Multivariate normal law and covariance convention: Multivariate normal law, including singular covariance.

Positive-definite multivariate normal density: A positive-definite multivariate normal law has its Gaussian density.

Lebesgue measure and its measurable sets: Lebesgue measurable sets, the family $\mathcal{L}({\mathbb{R}}^n)$, and the restricted set function ${\lambda }_n$.

Finite product measure: For sigma-finite factors, the product measure exists, has the rectangle formula, is sigma-finite, and is unique.

Iterated integration for nonnegative products: Tonelli's theorem for nonnegative measurable functions on a sigma-finite product.

Product Euclidean Lebesgue measure: On Borel subsets of R^{m+n}, the product lambda_m times lambda_n agrees with lambda_{m+n}.

Forward chain and reverse model: Ho, Jain, and Abbeel, §§2–3.

Forward Gaussian kernels: Prince, Chapter 18, “The diffusion kernel”.