Energy models, score matching, and contrastive divergence

Define $\varphi (t):=\exp (-t)$. By [F4], $\varphi (t)={\sum }_{n\ge 0}(-1{)}^n{t}^n\mathrm{/}n!$ is a power-series sum with infinite radius, so [F5] makes $\varphi$ continuous on $\mathbb{R}$. Thus [F6] makes it Borel measurable, and [F7] with the measurable energy from [F1] makes $x\mapsto \exp (-E_{\theta }(x))$ measurable. Its values are positive by [F8], hence nonnegative. Put $M_{\theta }(A):={\int }_A{e}^{-E_{\theta }}d\mu$. Now [F2] makes $M_{\theta }$ a measure and $M_{\theta }(X)=Z_{\theta }$. Thus $P_{\theta }(\mathrm{\varnothing })=Z_{\theta }^{-1}{M}_{\theta }(\mathrm{\varnothing })=0$. For any pairwise disjoint sequence $(A_n{)}_{n\ge 0}$, [F2] and finite-sum homogeneity give $P_{\theta }({\bigcup }_{n\ge 0}{A}_n)=Z_{\theta }^{-1}{\sum }_{n\ge 0}{M}_{\theta }(A_n)={\lim }_{N\to \mathrm{\infty }}{\sum }_{n=0}^N{Z}_{\theta }^{-1}{M}_{\theta }(A_n)={\sum }_{n\ge 0}{P}_{\theta }(A_n)$. Therefore $P_{\theta }$ is a measure.

Its total mass is $P_{\theta }(X)=Z_{\theta }^{-1}{M}_{\theta }(X)=1$, so [F3] makes $P_{\theta }$ a probability measure.

By [F1], the admissible model has density $e^{-E_{\theta }}\mathrm{/}{Z}_{\theta }$, with $Z_{\theta }$ independent of $x$. Since $p$ is $C^1$, [F6, F23] make it continuous; [F9, F10] then make it Lebesgue measurable, so [F2] makes $P_p$ a measure, and [F5] makes it a probability measure. The local increment bound [F23] gives, at each $x\in U=\{p>0\}$, constants $C_x,{\delta }_x>0$ with $\mathrm{\mid }p(y)-p(x)\mathrm{\mid }\le C_x\mathrm{\parallel }y-x\mathrm{\parallel }$ for $\mathrm{\parallel }y-x\mathrm{\parallel }<{\delta }_x$; taking $\mathrm{\parallel }y-x\mathrm{\parallel }<\min \{{\delta }_x,p(x)\mathrm{/}(2(C_x+1))\}$ shows $p(y)>0$, so $U$ is open. On $U$, [F6, F15, F24] make $s_p=\mathrm{\nabla }\log p$ continuous. For each open $O\subseteq {\mathbb{R}}^d$, its score preimage is the union of the relative-open preimage in $U$ and $U^c$ when $0\in O$; hence the zero extension is Borel and Lebesgue measurable by [F10]. By [F8], weak-derivative representatives agree off a Lebesgue-null set; [F2] makes every such set $P_p$-null, so the divergence integral is representative-independent. Choose $\chi$ from [F11] with $r=1$ and $R=2$. By [F9, F10], this continuous bump is Lebesgue measurable. Its gradient vanishes outside $B_2$; [F12, F13] make $C:=\mathrm{\parallel }\mathrm{\nabla }\chi {\mathrm{\parallel }}_{\mathrm{\infty }}$ finite. The box $[-1\mathrm{/}(2d),1\mathrm{/}(2d){)}^d$ lies in $B_1$, since its points have norm at most $\sqrt{d}\mathrm{/}(2d)\le 1\mathrm{/}2$, while $\mathrm{supp}\chi \subset B_2\subset [-3,3{)}^d$ and $0\le \chi \le 1$. Thus [F3, F16] give $d^{-d}\le M:=\int \chi d{\lambda }_d\le 6^d$. Set ${\rho }_0=\chi \mathrm{/}M$; positive homogeneity [F16] gives it unit mass. Let ${\rho }_{0,\epsilon }$ be its scaled family from [F18]; [F18] gives support in ${\bar{B}}_{2\epsilon }$ and [F17] gives unit mass. For $R>0$, define ${\chi }_R(x)=\chi (x\mathrm{/}R)$. Then ${\chi }_R=1$ on ${\bar{B}}_R$, vanishes outside $B_{2R}$, and [F24] gives $\mathrm{\parallel }\mathrm{\nabla }{\chi }_R{\mathrm{\parallel }}_{\mathrm{\infty }}\le C\mathrm{/}R$.

Fix $R$ and put $\phi =p{\chi }_R$. The product rule [F14] makes $\phi$ a nonnegative $C^1$ function supported in ${\bar{B}}_{2R}$. By [F12, F13], $\phi$ and its first derivatives are bounded there; the ball lies in a finite-volume box by [F3], so $\phi$ is locally integrable. The convolution ${\phi }_{\epsilon }=\phi \ast {\rho }_{0,\epsilon }$ is smooth by [F19] and supported in ${\bar{B}}_{2R+2}$ for $\epsilon \le 1$, since $\mathrm{supp}\phi \subseteq {\bar{B}}_{2R}$ and $\mathrm{supp}{\rho }_{0,\epsilon }\subseteq {\bar{B}}_2$. By [F17], applied to the nonnegative convolution integrand, ${\phi }_{\epsilon }(x)=\int \phi (x-z){\rho }_{0,\epsilon }(z)dz$. For each coordinate $j$, [F24] gives the parameter derivative of $f(z,t)=\phi (x+t{e}_j-z){\rho }_{0,\epsilon }(z)$. Its sections are integrable because $\phi$ is bounded and the kernel has mass one; its derivative is measurable and bounded by $\mathrm{\parallel }{\mathrm{\partial }}_j\phi {\mathrm{\parallel }}_{\mathrm{\infty }}{\rho }_{0,\epsilon }(z)$, an integrable majorant. Thus [F20] gives ${\mathrm{\partial }}_j{\phi }_{\epsilon }(x)=\int {\mathrm{\partial }}_j\phi (x-z){\rho }_{0,\epsilon }(z)dz$. The unit mass and shrinking support give, for each $x$, $\mathrm{\mid }{\phi }_{\epsilon }(x)-\phi (x)\mathrm{\mid }\le {\sup }_{\mathrm{\parallel }z\mathrm{\parallel }\le 2\epsilon }\mathrm{\mid }\phi (x-z)-\phi (x)\mathrm{\mid }\to 0$, with the same estimate for each first derivative. Since $\phi$ and $\mathrm{\nabla }\phi$ are bounded by [F12, F13], the weak-identity integrands are dominated on this common compact support by constants times $\mathrm{\mid }{\psi }_{\theta ,i}\mathrm{\mid }$ and $\mathrm{\mid }{g}_{ii}\mathrm{\mid }$. Local integrability and [F4] pass the identities from ${\phi }_{\epsilon }$ to $\phi$. Summing over $i$ and expanding $\mathrm{\nabla }(p{\chi }_R)$ with [F14] yields $\int {\chi }_R{\psi }_{\theta }\cdot \mathrm{\nabla }pd{\lambda }_d+\int p{\psi }_{\theta }\cdot \mathrm{\nabla }{\chi }_{R}d{\lambda }_d=-\int p{\chi }_R\mathrm{div}{\psi }_{\theta }d{\lambda }_d$.

On $\{p>0\}$, [F15, F24] give $\mathrm{\nabla }p=p{s}_p$. At any point where $p=0$, nonnegativity makes it an interior local minimum, so [F21, F22] give $\mathrm{\nabla }p=0$ there too. Thus $\mathrm{\nabla }p=p{s}_p$ everywhere. Componentwise $\mathrm{\mid }{\psi }_{\theta }\cdot s_p\mathrm{\mid }\le {\sum }_i\mathrm{\mid }{\psi }_{\theta ,i}{s}_{p,i}\mathrm{\mid }$ and $2\mathrm{\mid }ab\mathrm{\mid }\le a^2+b^2$ imply $\int \mathrm{\mid }{\psi }_{\theta }\cdot \mathrm{\nabla }p\mathrm{\mid }d{\lambda }_d\le \frac{1}{2}{\mathbb{E}}_p(\mathrm{\parallel }{\psi }_{\theta }{\mathrm{\parallel }}^2+\mathrm{\parallel }{s}_p{\mathrm{\parallel }}^2)<\mathrm{\infty }$ and $\int p\mathrm{\mid }\mathrm{div}{\psi }_{\theta }\mathrm{\mid }d{\lambda }_d<\mathrm{\infty }$. Also $\mathrm{\parallel }{\psi }_{\theta }\mathrm{\parallel }\le (1+\mathrm{\parallel }{\psi }_{\theta }{\mathrm{\parallel }}^2)\mathrm{/}2$, so the cutoff term from step 1.1 is bounded in absolute value by $\frac{C}{2R}(1+{\mathbb{E}}_p\mathrm{\parallel }{\psi }_{\theta }{\mathrm{\parallel }}^2)\to 0$.

Let $R=n$ for integers $n\to \mathrm{\infty }$. Then ${\chi }_n\to 1$ pointwise and $\mathrm{\mid }{\chi }_n\mathrm{\mid }\le 1$. The first integrand in step 2.1 is dominated by $\mathrm{\mid }{\psi }_{\theta }\cdot \mathrm{\nabla }p\mathrm{\mid }$ and the third by $p\mathrm{\mid }\mathrm{div}{\psi }_{\theta }\mathrm{\mid }$, both integrable by step 2.2. Applying [F4] to both terms and using the vanishing cutoff term from step 2.2 gives $\int {\psi }_{\theta }\cdot \mathrm{\nabla }pd{\lambda }_d=-\int p\mathrm{div}{\psi }_{\theta }d{\lambda }_d$.

Expanding the square gives $J(\theta )=\frac{1}{2}{\mathbb{E}}_p\mathrm{\parallel }{\psi }_{\theta }{\mathrm{\parallel }}^2+\frac{1}{2}{\mathbb{E}}_p\mathrm{\parallel }{s}_p{\mathrm{\parallel }}^2-\int p{\psi }_{\theta }\cdot s_{p}d{\lambda }_d$. Since $\mathrm{\nabla }p=p{s}_p$ by step 2.2, step 3.1 turns the last term into $+{\mathbb{E}}_p[\mathrm{div}{\psi }_{\theta }]$; the remaining data-score term is $C_p$, independent of $\theta$.

Fix a coordinate $j$ and vary only ${\theta }_j$ in a small interval inside the open set $\mathrm{\Theta }$. Since $\mathcal{X}$ is finite and each energy is real, [F13] makes every $e^{-E_{\theta }(x)}$ positive; hence [F1] gives $0<Z_{\theta }<\mathrm{\infty }$. The formula for $p_{\theta }$ then gives positive coordinates summing to one, so it is a probability vector by [F4]. By [F6], [F7] and [F8], differentiating the finite sum gives ${\mathrm{\partial }}_j{Z}_{\theta }=-{\sum }_x{e}^{-E_{\theta }(x)}{\mathrm{\partial }}_j{E}_{\theta }(x)$. The same chain and quotient rules give ${\mathrm{\partial }}_j{p}_{\theta }(x)\mathrm{/}{p}_{\theta }(x)=-{\mathrm{\partial }}_j{E}_{\theta }(x)-{\mathrm{\partial }}_j{Z}_{\theta }\mathrm{/}{Z}_{\theta }=-{\mathrm{\partial }}_j{E}_{\theta }(x)+{\sum }_y{p}_{\theta }(y){\mathrm{\partial }}_j{E}_{\theta }(y)$; [F9] identifies the left side with ${\mathrm{\partial }}_j\log p_{\theta }(x)$. Because $q_0$ is fixed and sums to one by [F4], differentiating $\mathcal{L}=-{\sum }_x{q}_0(x)\log p_{\theta }(x)$ and summing this identity yields the displayed likelihood-gradient formula.

The recurrence in [F2] gives $q_{j+1,\theta }=q_{j,\theta }{K}_{\theta }$, hence $q_{k,\theta }=q_0{K}_{\theta }^k$ and ${\sum }_x{q}_{k,\theta }(x)=1$. For $k=1$ the product rule [F6] gives ${\mathrm{\partial }}_j(q_0{K}_{\theta })=q_0{\mathrm{\partial }}_j{K}_{\theta }$. If ${\mathrm{\partial }}_j(q_0{K}_{\theta }^k)={\sum }_{s=0}^{k-1}{q}_0{K}_{\theta }^s({\mathrm{\partial }}_j{K}_{\theta })K_{\theta }^{k-1-s}$, then entrywise application of [F6] to $q_0{K}_{\theta }^k{K}_{\theta }$ yields ${\sum }_{s=0}^{k-1}{q}_0{K}_{\theta }^s({\mathrm{\partial }}_j{K}_{\theta })K_{\theta }^{k-s}+q_0{K}_{\theta }^k({\mathrm{\partial }}_j{K}_{\theta })$, which is the same formula with $k+1$ by appending the $s=k$ term. Thus the stated product derivative holds for every $k$. Differentiating ${\sum }_x{q}_{k,\theta }(x)=1$ by [F6] gives ${\sum }_x{\mathrm{\partial }}_j{q}_{k,\theta }(x)=0$. Strict positivity of $K_{\theta }$ and nonzero mass of $q_0$ imply $q_{k,\theta }(x)>0$ for every $x$, so the following logarithms are finite.

Put $w(x_0,\dots ,x_k):=q_0(x_0){\prod }_{s=0}^{k-1}{K}_{\theta }(x_s,x_{s+1})$. These weights are nonnegative. Summing successively over $x_k,\dots ,x_1$ and using the row sums of $K_{\theta }$ reduces ${\sum }_{\omega \in \mathrm{\Omega }}w(\omega )$ to ${\sum }_{x_0}{q}_0(x_0)=1$. The set function ${\mathbb{P}}_{\theta }(A)={\sum }_{\omega \in A}w(\omega )$ vanishes on the empty set and is countably additive: a disjoint sequence of subsets of finite $\mathrm{\Omega }$ has only finitely many nonempty terms. Thus [F10] makes it a measure, and [F11] makes it a probability measure.

By the quotient rule [F6], ${\mathrm{\partial }}_j(q_{k,\theta }(x)\mathrm{/}{p}_{\theta }(x))={\mathrm{\partial }}_j{q}_{k,\theta }(x)\mathrm{/}{p}_{\theta }(x)-q_{k,\theta }(x){\mathrm{\partial }}_j{p}_{\theta }(x)\mathrm{/}{p}_{\theta }(x{)}^2$; the chain and logarithm derivative rules [F7, F9] then give ${\mathrm{\partial }}_j\log (q_{k,\theta }(x)\mathrm{/}{p}_{\theta }(x))={\mathrm{\partial }}_j{q}_{k,\theta }(x)\mathrm{/}{q}_{k,\theta }(x)-{\mathrm{\partial }}_j\log p_{\theta }(x)$. Applying the product rule [F6] to each summand of $D_{\mathrm{K}\mathrm{L}}(q_{k,\theta }\mathrm{\parallel }{p}_{\theta })$ therefore gives ${\mathrm{\partial }}_j{q}_{k,\theta }(x)(\log (q_{k,\theta }(x)\mathrm{/}{p}_{\theta }(x))+1)-q_{k,\theta }(x){\mathrm{\partial }}_j\log p_{\theta }(x)$. Summing the finite terms, the $+1$ contribution vanishes by step 1.2. Substitution from step 1.1 yields ${\mathrm{\partial }}_j{D}_{\mathrm{K}\mathrm{L}}(q_{k,\theta }\mathrm{\parallel }{p}_{\theta })={\sum }_x{\mathrm{\partial }}_j{q}_{k,\theta }(x)\log (q_{k,\theta }(x)\mathrm{/}{p}_{\theta }(x))+{\sum }_x{q}_{k,\theta }(x){\mathrm{\partial }}_j{E}_{\theta }(x)-{\sum }_x{p}_{\theta }(x){\mathrm{\partial }}_j{E}_{\theta }(x)$.

For $s=0$, $K_{\theta }^0=I$ is row-stochastic and its entries are the length-zero path weights. If the entries of $K_{\theta }^s$ sum the products along all length-$s$ paths, then $K_{\theta }^{s+1}(x,y)={\sum }_z{K}_{\theta }^s(x,z)K_{\theta }(z,y)$ appends the final transition, so its entries sum the length-$(s+1)$ path weights; also ${\sum }_y{K}_{\theta }^{s+1}(x,y)={\sum }_z{K}_{\theta }^s(x,z){\sum }_y{K}_{\theta }(z,y)=1$. Thus ${\mathbb{P}}_{\theta }(X_0=x,X_k=y)=q_0(x)K_{\theta }^k(x,y)$. The function $g_f$ is finite-valued; for each Borel $C\subseteq \mathbb{R}$, $\{g_f(X_0)\in C\}=X_0^{-1}(\{x:g_f(x)\in C\})$, so $g_f(X_0)$ is $\sigma (X_0)$-measurable. Both $f(X_k)$ and $g_f(X_0)$ are integrable since $\mathrm{\Omega }$ is finite.

Since $q_0$ is fixed, the terms $q_0(x)\log q_0(x)$ in $D_{\mathrm{K}\mathrm{L}}(q_0\mathrm{\parallel }{p}_{\theta })$ have zero derivative; applying the same product and logarithm rules [F6, F9] and using the log-density formula from step 1.1 and ${\sum }_x{q}_0(x)=1$ gives ${\mathrm{\partial }}_j{D}_{\mathrm{K}\mathrm{L}}(q_0\mathrm{\parallel }{p}_{\theta })={\sum }_x{q}_0(x){\mathrm{\partial }}_j{E}_{\theta }(x)-{\sum }_x{p}_{\theta }(x){\mathrm{\partial }}_j{E}_{\theta }(x)$. Subtracting step 2.1 cancels the model expectations and yields the stated ${\mathrm{\partial }}_j{\mathrm{CD}}_k$ formula. By [F3], the algorithm uses only its first two terms; the remaining sum is the derivative through $K_{\theta }^k$. Comparing with step 1.1 also shows that the algorithm replaces the exact likelihood negative phase $p_{\theta }$ by $q_{k,\theta }$.

Every event in $\sigma (X_0)$ has the form $\{X_0\in B\}$ for some $B\subseteq \mathcal{X}$, since these preimages form a sigma-algebra and generate $\sigma (X_0)$. For every such event, step 2.2 gives $\int {\mathbf{1}}_{\{X_0\in B\}}f(X_k)d{\mathbb{P}}_{\theta }={\sum }_{x\in B}{q}_0(x){\sum }_y{K}_{\theta }^k(x,y)f(y)=\int {\mathbf{1}}_{\{X_0\in B\}}{g}_f(X_0)d{\mathbb{P}}_{\theta },$ where terms with $q_0(x)=0$ vanish. The integrability and measurability from step 2.2 and this event-integral identity satisfy [F12], so $g_f(X_0)$ is the asserted conditional-expectation version.