DDPM variational objective and weighted noise-prediction terms

The reverse path density is strictly positive and finite at every finite path because every Gaussian variance is positive and each mean is finite. Its path marginal on $x_{1:T}$ is a probability density by successive integration of its reverse kernels and prior. For fixed $x_0$, the decoder density satisfies $p_{\theta }(x_0\mid x_1)\le (2\pi {\sigma }_1^2{)}^{-d\mathrm{/}2}$, so integrating against that path marginal gives $p_{\theta }(x_0)<\mathrm{\infty }$. It is strictly positive: on the positive-measure box $[-1,1{]}^{dT}$, the sets $\{p_{\theta }(x_{0:T})\ge 1\mathrm{/}n\}$ for positive integers $n$ cover the box because the density is positive everywhere. If all these sets had measure zero, their countable union would have measure zero, contradicting the positive measure of the box; one set therefore has positive measure, and the integral is positive. Thus ${\pi }_{\theta }(\cdot \mid x_0)$ is a proper positive density.

The forward path density is a probability density and is positive everywhere. Its log is the finite sum $\log q(x_{1:T}\mid x_0)={\sum }_{t=1}^T[-\frac{d}{2}\log (2\pi {\beta }_t)-\frac{\mathrm{\parallel }{x}_t-\sqrt{{\alpha }_t}{x}_{t-1}{\mathrm{\parallel }}_2^2}{2{\beta }_t}]$. Under $q$, each normalized residual $(x_t-\sqrt{{\alpha }_t}{x}_{t-1})\mathrm{/}\sqrt{{\beta }_t}$ has law $N_d(0,I_d)$ and finite second moment $d$ by [F10], so ${\mathbb{E}}_q\mathrm{\mid }\log q\mathrm{\mid }<\mathrm{\infty }$. Since the reverse density is also positive everywhere, ${\pi }_{\theta }>0$ everywhere and the support condition $Q_{x_0}\ll {\mathrm{\Pi }}_{\theta ,x_0}$ holds.

Let $\gamma (x_{1:T})=p_{\theta }(x_{0:T})$ on path space with base measure ${\lambda }_{dT}$. Step 1.1 gives $Z=\int \gamma d{\lambda }_{dT}=p_{\theta }(x_0)\in (0,\mathrm{\infty })$ and the normalized target $\pi =\gamma \mathrm{/}Z$; step 1.2 gives the support condition, and the Statement's integrability hypothesis together with step 1.2 gives ${\mathbb{E}}_q(\mathrm{\mid }\log \gamma \mathrm{\mid }+\mathrm{\mid }\log q\mathrm{\mid })<\mathrm{\infty }$. The direction is $Q_{x_0}\ll {\mathrm{\Pi }}_{\theta ,x_0}$, as required by [F8]. Therefore [F3], [F6], and [F9] apply, and [F4] gives the exact log-evidence identity and nonnegative reverse KL.

For $t\ge 2$, the Markov property and positive Gaussian densities give $q(x_t\mid x_{t-1})=q(x_{t-1}\mid x_t,x_0)q(x_t\mid x_0)\mathrm{/}q(x_{t-1}\mid x_0)$. Substitute this into $p_{\theta }(x_{0:T})\mathrm{/}q(x_{1:T}\mid x_0)$. The ratios $q(x_{t-1}\mid x_0)\mathrm{/}q(x_t\mid x_0)$ telescope, leaving $p_{\theta }(x_{0:T})\mathrm{/}q(x_{1:T}\mid x_0)=[p(x_T)\mathrm{/}q(x_T\mid x_0)]p_{\theta }(x_0\mid x_1){\prod }_{t=2}^T{p}_{\theta }(x_{t-1}\mid x_t)\mathrm{/}q(x_{t-1}\mid x_t,x_0)$. Each reverse Gaussian log-density factor is bounded above; the assumed integrability of their finite sum therefore makes each factor integrable. The forward marginals and conditioned laws are Gaussian with finite quadratic moments, so their log-density terms are integrable as well. Taking expectations term by term now gives the stated endpoint-plus-step decomposition, with the $t=1$ term $-{\mathbb{E}}_{q(x_1\mid x_0)}\log p_{\theta }(x_0\mid x_1)$.

For $t\ge 2$, substitute $x_t=\sqrt{{\bar{\alpha }}_t}{x}_0+\sqrt{1-{\bar{\alpha }}_t}ϵ$ into the posterior mean in [F2]. Using ${\bar{\alpha }}_t={\alpha }_t{\bar{\alpha }}_{t-1}$ and ${\beta }_t=1-{\alpha }_t$ gives ${\tilde{\mu }}_t(x_t,x_0)=\frac{1}{\sqrt{{\alpha }_t}}(x_t-\frac{{\beta }_t}{\sqrt{1-{\bar{\alpha }}_t}}ϵ)$. The reverse mean is the same expression with ${ϵ}_{\theta }(x_t,t)$ in place of $ϵ$, so $\mathrm{\parallel }{\tilde{\mu }}_t-{\mu }_{\theta ,t}{\mathrm{\parallel }}_2^2=\frac{{\beta }_t^2}{{\alpha }_t(1-{\bar{\alpha }}_t)}\mathrm{\parallel }ϵ-{ϵ}_{\theta }(x_t,t){\mathrm{\parallel }}_2^2$. Both posterior and reverse variances are positive for $t\ge 2$, so [F5] applies with $v={\tilde{\beta }}_t$ and $w={\sigma }_t^2$; it gives exactly the coefficient $w_t$ and constant $C_t$ in the Statement.

The terminal term remains $D_{\mathrm{K}\mathrm{L}}(q(x_T\mid x_0)\mathrm{\parallel }{N}_d(0,I_d))$ in step 2.2 because finite $T$ and $0<{\beta }_t<1$ give ${\bar{\alpha }}_T>0$; the $t=1$ term remains the separate positive-variance decoder log-likelihood. For an explicit counterexample, take $d=1$, $T=2$, ${\beta }_1=\frac{1}{4}$, ${\beta }_2=\frac{1}{2}$, ${\sigma }_1^2=(2\pi {)}^{-1}$, and ${\sigma }_2^2={\tilde{\beta }}_2=\frac{1}{5}$. Then ${\bar{\alpha }}_1=\frac{3}{4}$, ${\bar{\alpha }}_2=\frac{3}{8}$, $w_2=\frac{(1\mathrm{/}2{)}^2}{2(1\mathrm{/}5)(1\mathrm{/}2)(5\mathrm{/}8)}=2$, and $C_2=\frac{1}{2}(1-1+\log 1)=0$. For a fixed datum $x_0$, set ${ϵ}_{\theta }(x_1,1)=(x_1-\sqrt{3\mathrm{/}4}{x}_0)\mathrm{/}\sqrt{1\mathrm{/}4}$ and ${ϵ}_{\theta }(x_2,2)=(x_2-\sqrt{3\mathrm{/}8}{x}_0)\mathrm{/}\sqrt{5\mathrm{/}8}+c$, with any $c\in \mathbb{R}$. These are finite Borel maps, and their Gaussian log densities have finite $q$-expectations because they are quadratic functions of finite-dimensional Gaussian variables. The first map makes ${\mu }_{\theta ,1}(x_1)=x_0$, so the decoder density at its mean is $(2\pi {\sigma }_1^2{)}^{-1\mathrm{/}2}=1$; integrating it against the normalized reverse path density gives $p_{\theta }(x_0)=1$ for every $c$. Under the forward chain, the $t=1$ noise error is zero and the $t=2$ error is $c$, so $L_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}=c^2\mathrm{/}2$, while the exact weighted $t=2$ ELBO contribution is $2c^2$ and its prior term is independent of $c$. Taking $c\mathrm{\ne }0$ gives $L_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}}>0$ but exact negative log-likelihood $-\log p_{\theta }(x_0)=0$. Thus the unweighted loss is a surrogate, not in general the maximum-likelihood objective.