Discrete diffusion chains and DDPM objectives

Assume AC. Let $d,T\ge 1$ be finite integers, let $I_d$ be the $d\times d$ identity matrix, and let ${\beta }_1,\dots ,{\beta }_T\in (0,1)$. Set ${\alpha }_t=1-{\beta }_t$, ${\bar{\alpha }}_0=1$, and ${\bar{\alpha }}_t={\prod }_{s=1}^t{\alpha }_s$. Write ${\lambda }_d$ for Lebesgue measure on ${\mathbb{R}}^d$ and define, for $v>0$, ${\varphi }_{d,v}(y;m)=(2\pi v{)}^{-d\mathrm{/}2}\exp \text{\hspace{0.17em}⁣}(-\frac{\mathrm{\parallel }y-m{\mathrm{\parallel }}_2^2}{2v}),y,m\in {\mathbb{R}}^d\mathrm{.}$ The notation $\mathrm{\parallel }z{\mathrm{\parallel }}_2^2={\sum }_{i=1}^d{z}_i^2$ is the squared Euclidean norm. The displayed function is the Lebesgue density of $N_d(m,v{I}_d)$.

For each fixed $x_0\in {\mathbb{R}}^d$, the forward chain has path density on $({\mathbb{R}}^d{)}^T$ $q(x_{1:T}\mid x_0)={\prod }_{t=1}^T{\varphi }_{d,{\beta }_t}(x_t;\sqrt{{\alpha }_t}{x}_{t-1})\mathrm{.}$ Thus its transition from $x_{t-1}$ to $x_t$ is Gaussian with mean $\sqrt{{\alpha }_t}{x}_{t-1}$ and covariance ${\beta }_t{I}_d$. Each factor is a normalized density, and finite successive integration of the nonnegative product gives a probability density.

For a generic adjacent pair, the two model directions are shown below. The arrows indicate the direction in which each conditional density generates its target state; they assert no equality or inverse relation between the kernels.

diagram failed to render:
\begin{tikzpicture}[>=stealth]
  \node (previous) at (0,0) {$x_{t-1}$};
  \node (current) at (4,0) {$x_t$};
  \draw[->] (previous) to[bend left=24] node[midway,above] {$q(x_t\mid x_{t-1})$} (current);
  \draw[->] (current) to[bend left=24] node[midway,below] {$p_{\theta,t}(x_{t-1}\mid x_t)$} (previous);
\end{tikzpicture}

A learned reverse Gaussian model consists of finite-valued Borel maps ${\mu }_{\theta ,t}:{\mathbb{R}}^d\to {\mathbb{R}}^d$ and fixed variances ${\sigma }_t^2\in (0,\mathrm{\infty })$, for $1\le t\le T$. It is the joint density $p_{\theta }(x_{0:T})={\varphi }_{d,1}(x_T;0){\prod }_{t=1}^T{\varphi }_{d,{\sigma }_t^2}(x_{t-1};{\mu }_{\theta ,t}(x_t))\mathrm{.}$ Its prior is $p(x_T)=N_d(0,I_d)$, and its marginal density at $x_0$ is $p_{\theta }(x_0)={\int }_{({\mathbb{R}}^d{)}^T}{p}_{\theta }(x_{0:T})d{\lambda }_d(x_1)\cdots d{\lambda }_d(x_T),$ whenever this integral is finite. The joint reverse density is normalized by successive integration of its Gaussian transition factors followed by the prior density.

For $t=1$, the forward-chain conditional $q(x_0\mid x_1,x_0)$ assigns probability one to every Borel set containing $x_0$ and zero to every Borel set not containing $x_0$; it is not a Lebesgue density. The separate learned reverse likelihood $p_{\theta }(x_0\mid x_1)$ is the Gaussian density above with variance ${\sigma }_1^2>0$.

Multivariate normal law and covariance convention: Multivariate normal law, including singular covariance.

Positive-definite multivariate normal density: A positive-definite multivariate normal law has its Gaussian density.

Lebesgue measure and its measurable sets: Lebesgue measurable sets, the family $\mathcal{L}({\mathbb{R}}^n)$, and the restricted set function ${\lambda }_n$.

Finite product measure: For sigma-finite factors, the product measure exists, has the rectangle formula, is sigma-finite, and is unique.

Iterated integration for nonnegative products: Tonelli's theorem for nonnegative measurable functions on a sigma-finite product.

Product Euclidean Lebesgue measure: On Borel subsets of R^{m+n}, the product lambda_m times lambda_n agrees with lambda_{m+n}.

Forward chain and reverse model: Ho, Jain, and Abbeel, §§2–3.

Forward Gaussian kernels: Prince, Chapter 18, “The diffusion kernel”.

Assume AC. Let $d\ge 1$, $m,n\in {\mathbb{R}}^d$, and $v,w>0$. If $Q=N_d(m,v{I}_d)$ and $P=N_d(n,w{I}_d)$, then $D_{\mathrm{K}\mathrm{L}}(Q\mathrm{\parallel }P)=\frac{d}{2}(\frac{v}{w}-1+\log \frac{w}{v})+\frac{\mathrm{\parallel }m-n{\mathrm{\parallel }}_2^2}{2w},$ where the KL divergence is directional, its first argument is absolutely continuous with respect to its second, and $\log$ is the natural logarithm.

Gaussian density formula: A positive-definite multivariate normal law has its Gaussian density.

Directional KL convention: Kullback–Leibler divergence.

Natural logarithm: The natural logarithm as the inverse of the exponential function.

Gaussian reverse-KL terms in diffusion objectives: Ho, Jain, and Abbeel, §3.2.

Gaussian reverse-process loss: Prince, Chapter 18, “The reverse process and the tractable conditional”.

Expectation and covariance convention: Multivariate normal law, including singular covariance.

Natural logarithm: The natural logarithm as the inverse of the exponential function.

Assume AC and the finite Gaussian forward chain with dimension $d\ge 1$, horizon $T\ge 1$, and schedule $0<{\beta }_t<1$ from the preceding definition. For each $x_0\in {\mathbb{R}}^d$ and $1\le t\le T$, $q(x_t\mid x_0)=N_d\text{\hspace{0.17em}⁣}(\sqrt{{\bar{\alpha }}_t}{x}_0,(1-{\bar{\alpha }}_t)I_d)\mathrm{.}$ For every $2\le t\le T$, the conditional law given both endpoints is $q(x_{t-1}\mid x_t,x_0)=N_d\text{\hspace{0.17em}⁣}(\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}{x}_0+\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})}{1-{\bar{\alpha }}_t}{x}_t,{\tilde{\beta }}_t{I}_d),{\tilde{\beta }}_t=\frac{{\beta }_t(1-{\bar{\alpha }}_{t-1})}{1-{\bar{\alpha }}_t}>0.$ The formula for $q(x_{t-1}\mid x_t,x_0)$ is only asserted for $t\ge 2$; at $t=1$ the conditioning on $x_0$ makes $q(x_0\mid x_1,x_0)$ a point mass, not a Lebesgue density. The Gaussian conditional above does not assert that the unconditional reverse law $q(x_{t-1}\mid x_t)$ is Gaussian.

Closed Gaussian forward marginal, posterior mean, and posterior variance: Ho, Jain, and Abbeel, Eqs. (4), (6), and (7).

Forward Gaussian composition and conditioned posterior: Prince, Chapter 18, “The diffusion kernel” and “The tractable conditional”.

Joint Gaussian conditional law and its positive-definite observed covariance hypothesis: Conditional law of a jointly Gaussian block with positive-definite observed covariance.

Finite independent Gaussian stack: A finite stack of independent Gaussian vectors is jointly Gaussian.

Assume AC. Fix a finite Gaussian forward chain and reverse model as defined above, with finite-valued Borel reverse means and fixed variances ${\sigma }_t^2\in (0,\mathrm{\infty })$. For the noise-prediction parameterization, let ${ϵ}_{\theta }:{\mathbb{R}}^d\times \{1,\dots ,T\}\to {\mathbb{R}}^d$ be Borel and set ${\mu }_{\theta ,t}(x_t)=\frac{1}{\sqrt{{\alpha }_t}}(x_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{ϵ}_{\theta }(x_t,t)),1\le t\le T\mathrm{.}$ For each fixed observed $x_0$, require $0<p_{\theta }(x_0)<\mathrm{\infty }$, define the proper posterior density ${\pi }_{\theta }(x_{1:T}\mid x_0)=\frac{p_{\theta }(x_{0:T})}{p_{\theta }(x_0)},$ and let $Q_{x_0}$ and ${\mathrm{\Pi }}_{\theta ,x_0}$ be the path probability measures with Lebesgue densities $q(x_{1:T}\mid x_0)$ and ${\pi }_{\theta }(x_{1:T}\mid x_0)$. Require $Q_{x_0}\ll {\mathrm{\Pi }}_{\theta ,x_0}$ and ${\mathbb{E}}_{q(\cdot \mid x_0)}\text{\hspace{0.17em}⁣}[\mathrm{\mid }\log q(x_{1:T}\mid x_0)\mathrm{\mid }+\mathrm{\mid }\log p_{\theta }(x_{0:T})\mathrm{\mid }]<\mathrm{\infty }\mathrm{.}$ Here ${\mathbb{E}}_q[f]$ means the integral of $f$ against the path law with density $q(\cdot \mid x_0)$. Define ${\mathrm{ELBO}}_{\theta }(x_0)={\mathbb{E}}_q\text{\hspace{0.17em}⁣}[\log p_{\theta }(x_{0:T})-\log q(x_{1:T}\mid x_0)]\mathrm{.}$ Then $\log p_{\theta }(x_0)={\mathrm{ELBO}}_{\theta }(x_0)+D_{\mathrm{K}\mathrm{L}}(Q_{x_0}\mathrm{\parallel }{\mathrm{\Pi }}_{\theta ,x_0}),D_{\mathrm{K}\mathrm{L}}(Q_{x_0}\mathrm{\parallel }{\mathrm{\Pi }}_{\theta ,x_0})\ge 0.$ The negative ELBO has the exact decomposition $-{\mathrm{ELBO}}_{\theta }(x_0)=D_{\mathrm{K}\mathrm{L}}\text{\hspace{0.17em}⁣}(q(x_T\mid x_0)\parallel N_d(0,I_d))+{\mathbb{E}}_{q(x_1\mid x_0)}[-\log p_{\theta }(x_0\mid x_1)]+{\sum }_{t=2}^T{\mathbb{E}}_{q(x_t\mid x_0)}\text{\hspace{0.17em}⁣}\left[D_{\mathrm{K}\mathrm{L}}\text{\hspace{0.17em}⁣}(q(x_{t-1}\mid x_t,x_0)\parallel p_{\theta }(x_{t-1}\mid x_t))\right]\mathrm{.}$ In particular, for $t\ge 2$, write ${\tilde{\mu }}_t(x_t,x_0)=\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}{x}_0+\frac{\sqrt{{\alpha }_t}(1-{\bar{\alpha }}_{t-1})}{1-{\bar{\alpha }}_t}{x}_t,w_t=\frac{{\beta }_t^2}{2{\sigma }_t^2{\alpha }_t(1-{\bar{\alpha }}_t)},C_t=\frac{d}{2}(\frac{{\tilde{\beta }}_t}{{\sigma }_t^2}-1+\log \frac{{\sigma }_t^2}{{\tilde{\beta }}_t})\mathrm{.}$ If $x_t=\sqrt{{\bar{\alpha }}_t}{x}_0+\sqrt{1-{\bar{\alpha }}_t}ϵ$ with $ϵ\sim N_d(0,I_d)$, then the corresponding per-step KL term is ${\mathbb{E}}_{ϵ}\text{\hspace{0.17em}⁣}[w_t{\parallel ϵ-{ϵ}_{\theta }(x_t,t)\parallel }_2^2+C_t]\mathrm{.}$ At $t=1$, the reconstruction term is the separate learned likelihood $p_{\theta }(x_0\mid x_1)$ with variance ${\sigma }_1^2>0$; no Lebesgue density for $q(x_0\mid x_1,x_0)$ is used. For finite $T$, the terminal-prior KL is retained; no equality $q(x_T\mid x_0)=N_d(0,I_d)$ is assumed. The common unweighted objective $L_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}(\theta ;x_0)=\frac{1}{T}{\sum }_{t=1}^T{\mathbb{E}}_{ϵ}\text{\hspace{0.17em}⁣}\left[{\parallel ϵ-{ϵ}_{\theta }\text{\hspace{0.17em}⁣}(\sqrt{{\bar{\alpha }}_t}{x}_0+\sqrt{1-{\bar{\alpha }}_t}ϵ,t)\parallel }_2^2\right]$ drops the schedule weights and replaces the separate endpoint terms with noise-prediction losses, so it is not in general the exact maximum-likelihood objective.

Finite-chain ELBO decomposition and exact Gaussian posterior: Ho, Jain, and Abbeel, §§2–3 and Appendix A, Eqs. (5)–(7), (17)–(22).

Weighted noise prediction and the unweighted simple objective: Ho, Jain, and Abbeel, §3.2 and §3.4, Eqs. (8)–(14).

Pedagogical ELBO and noise-prediction derivation; its simplification approximates the finite-horizon terminal-prior term by zero: Prince, Chapter 18, “The diffusion ELBO” and “Simplifying it”.

Variational-family support and log-integrability requirements: Variational family and evidence lower bound.

Exact ELBO/KL identity and its AC branch: Evidence lower bound is log evidence minus reverse KL.

Exact Gaussian posterior conditional and its positive observed-covariance hypothesis: Closed Gaussian forward marginal and conditioned posterior.

Isotropic Gaussian reverse-KL calculation: Reverse KL between isotropic Gaussian laws.

Fix finite integers $d,N,T\ge 1$, a data set $D=(x_0^{(1)},\dots ,x_0^{(N)})\in ({\mathbb{R}}^d{)}^N$, forward variances ${\beta }_1,\dots ,{\beta }_T\in (0,1)$, reverse variances ${\sigma }_1^2,\dots ,{\sigma }_T^2\in (0,\mathrm{\infty })$, a Borel map ${ϵ}_{\theta }:{\mathbb{R}}^d\times \{1,\dots ,T\}\to {\mathbb{R}}^d$, and a mode from $\{\mathtt{\text{weighted-elbo}},\mathtt{\text{simple}}\}$. Set ${\alpha }_t=1-{\beta }_t$ and ${\bar{\alpha }}_t={\prod }_{s=1}^t{\alpha }_s$. Use the noise-prediction means ${\mu }_{\theta ,t}(y)=\frac{1}{\sqrt{{\alpha }_t}}(y-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}{ϵ}_{\theta }(y,t)),1\le t\le T,$ and the reverse Gaussian model with these means, variances ${\sigma }_t^2$, and prior $N_d(0,I_d)$ to define $p_{\theta }$. In weighted-elbo mode, require for each $x_0^{(i)}$ that $0<p_{\theta }(x_0^{(i)})<\mathrm{\infty }$, the forward path law be absolutely continuous with respect to the normalized reverse posterior path law, and ${\mathbb{E}}_{q(\cdot \mid x_0^{(i)})}\text{\hspace{0.17em}⁣}[\mathrm{\mid }\log q(x_{1:T}\mid x_0^{(i)})\mathrm{\mid }+\mathrm{\mid }\log p_{\theta }(x_{0:T})\mathrm{\mid }]<\mathrm{\infty }\mathrm{.}$ In simple mode, require for every $i$ and $t$ that ${\mathbb{E}}_{ϵ\sim N_d(0,I_d)}\text{\hspace{0.17em}⁣}\left[{\parallel ϵ-{ϵ}_{\theta }\text{\hspace{0.17em}⁣}(\sqrt{{\bar{\alpha }}_t}{x}_0^{(i)}+\sqrt{1-{\bar{\alpha }}_t}ϵ,t)\parallel }_2^2\right]<\mathrm{\infty }\mathrm{.}$ This procedure returns one stochastic loss estimate; it does not prescribe a parameter optimizer or claim optimizer convergence.

1. Choose $I$ uniformly from $\{1,\dots ,N\}$, choose $J$ uniformly from $\{1,\dots ,T\}$ independently of $I$, and draw $ϵ\sim N_d(0,I_d)$ independently of both.

2. Set $x_J=\sqrt{{\bar{\alpha }}_J}{x}_0^{(I)}+\sqrt{1-{\bar{\alpha }}_J}ϵ\mathrm{.}$

3. If the mode is $\mathtt{\text{simple}}$, return ${\parallel ϵ-{ϵ}_{\theta }(x_J,J)\parallel }_2^2\mathrm{.}$

4. If the mode is $\mathtt{\text{weighted-elbo}}$, first compute the exact terminal-prior term $K_T(x_0^{(I)})=\frac{1}{2}[{\bar{\alpha }}_T\mathrm{\parallel }{x}_0^{(I)}{\mathrm{\parallel }}_2^2+d(-{\bar{\alpha }}_T-\log (1-{\bar{\alpha }}_T))]\mathrm{.}$ If $J=1$, return $K_T(x_0^{(I)})-T\log p_{\theta }(x_0^{(I)}\mid x_J)\mathrm{.}$ If $J\ge 2$, return $K_T(x_0^{(I)})+T[C_J+w_J{\parallel ϵ-{ϵ}_{\theta }(x_J,J)\parallel }_2^2],$ where $w_J$ and $C_J$ are the exact per-step coefficient and variance constant in the DDPM objective theorem.

Each branch returns one sampled objective contribution. The $J=1$ weighted branch uses the positive-variance learned reconstruction likelihood; it never evaluates a density for the degenerate forward conditional $q(x_0\mid x_1,x_0)$. This algorithm specifies the sampling procedure only; it makes no optimizer-convergence claim.

Finite Gaussian chain and reverse likelihood: Finite Gaussian forward chain and Gaussian reverse model.

Terminal KL and weighted per-step terms: DDPM variational objective and weighted noise-prediction terms.

Gaussian KL formula used for the terminal term: Reverse KL between isotropic Gaussian laws.

Stochastic epsilon-prediction objective and simple-loss convention: Ho, Jain, and Abbeel, Algorithm 1 and Eqs. (12)–(14).

Pedagogical treatment of weighted noise-prediction terms and the simplified training objective; its ELBO discussion approximates the terminal-prior term by zero: Prince, Chapter 18, “The diffusion ELBO” and “Simplifying it”.

Finite expectation and the exact objective estimated in each mode are proved in lem-diffusion-objective-estimator-expectation.

Expected output of the diffusion objective estimator ↗

Assume AC. Use the admissible inputs of the epsilon-prediction diffusion objective estimator, and let $Y$ be its returned real random variable. Its expectation is finite. In simple mode, $\mathbb{E}[Y]=\frac{1}{N}{\sum }_{i=1}^N{L}_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}(\theta ;x_0^{(i)})\mathrm{.}$ In weighted-elbo mode, $\mathbb{E}[Y]=\frac{1}{N}{\sum }_{i=1}^N[-{\mathrm{ELBO}}_{\theta }(x_0^{(i)})]\mathrm{.}$ Here the simple loss and ELBO use the same schedule, reverse model and predictor as the estimator. Both identities include $N=1$ and $T=1$.

Estimator inputs and sampling instructions: Epsilon-prediction diffusion objective estimator.

Definitions of the two objectives: DDPM variational objective and weighted noise-prediction terms.

Diffusion objective sampling and weighting: Ho, Jain, and Abbeel, Eqs. (5), (12), (14) and Algorithm 1.