Contrastive divergence with k transitions

Let $(\mathcal{X},\mathcal{A},\mu )$ be a measure space, let $r$ be a positive integer and $\theta \in {\mathbb{R}}^r$, and let $E_{\theta }:\mathcal{X}\to \mathbb{R}$ be measurable in $x$ and continuously differentiable in $\theta$. Define the energy model $P_{\theta }$ relative to $\mu$ only when $0<Z_{\theta }:={\int }_{\mathcal{X}}{e}^{-E_{\theta }(x)}d\mu (x)<\mathrm{\infty },$ with density $e^{-E_{\theta }}\mathrm{/}{Z}_{\theta }$. Let $Q_0$ be a data probability measure on $(\mathcal{X},\mathcal{A})$ and let $K_{\theta }$ be a probability kernel from $\mathcal{X}$ to itself which leaves $P_{\theta }$ invariant, meaning ${\int }_{\mathcal{X}}{K}_{\theta }(x,A)P_{\theta }(dx)=P_{\theta }(A)(A\in \mathcal{A})\mathrm{.}$ Let $\{\ast \}$ carry its full sigma-algebra, let $G_{0,\theta }(\ast ,A):=Q_0(A)$, and recursively compose the probability kernel $G_{j,\theta }$ from $\{\ast \}$ to $\mathcal{X}$ with $K_{\theta }$ to obtain $G_{j+1,\theta }$. Define $Q_{j,\theta }(A):=G_{j,\theta }(\ast ,A)$; thus $Q_{0,\theta }=Q_0$ and $Q_{j+1,\theta }(A)={\int }_{\mathcal{X}}{K}_{\theta }(x,A)Q_{j,\theta }(dx),A\in \mathcal{A}\mathrm{.}$ Finite probability-kernel composition makes each $Q_{j,\theta }$ a probability measure. For an integer $k\ge 1$, $Q_{k,\theta }$ is the $k$-step reconstruction law.

If $Q_0$ and $Q_{k,\theta }$ have specified nonnegative measurable densities $r_{0,\theta }$ and $r_{k,\theta }$ relative to $P_{\theta }$, meaning $Q_{j,\theta }(A)={\int }_A{r}_{j,\theta }d{P}_{\theta }$ for every $A\in \mathcal{A}$ for $j\in \{0,k\}$, and $\int \mathrm{\mid }{r}_{j,\theta }\log r_{j,\theta }\mathrm{\mid }d{P}_{\theta }<\mathrm{\infty }(j\in \{0,k\}),$ define their finite contrastive-divergence objective by ${\mathrm{CD}}_k(\theta ):=\int r_{0,\theta }\log r_{0,\theta }d{P}_{\theta }-\int r_{k,\theta }\log r_{k,\theta }d{P}_{\theta },$ where $0\log 0:=0$ and $\log$ is the natural logarithm. On a finite state space where $P_{\theta }$ has positive mass at every state, write $p_{\theta }(x):=P_{\theta }(\{x\})$ and $q_{j,\theta }(x):=Q_{j,\theta }(\{x\})$. Then $r_{j,\theta }(x)=q_{j,\theta }(x)\mathrm{/}{p}_{\theta }(x)$ and the objective equals ${\sum }_{x:q_{0,\theta }(x)>0}{q}_{0,\theta }(x)\log \frac{q_{0,\theta }(x)}{p_{\theta }(x)}-{\sum }_{x:q_{k,\theta }(x)>0}{q}_{k,\theta }(x)\log \frac{q_{k,\theta }(x)}{p_{\theta }(x)},$ since each finite-space integral is the sum of the integrand times the point mass and $p_{\theta }{r}_{j,\theta }=q_{j,\theta }$.

For a positive integer $m$, the procedure takes independent data draws $x^1,\dots ,x^m$ with common law $Q_0$, a transition sampler for $K_{\theta }$, and a learning rate $\eta >0$; the reconstruction chains are drawn independently conditional on their starts. Its output is the updated parameter from the stop-gradient CD-$k$ update:

1. For each $i$, set $x^{i,0}=x^i$ and draw $x^{i,j+1}$ from $K_{\theta }(x^{i,j},\cdot )$ for $j=0,\dots ,k-1$.
2. Form ${\widehat{g}}_k=\frac{1}{m}{\sum }_{i=1}^m({\mathrm{\nabla }}_{\theta }{E}_{\theta }(x^i)-{\mathrm{\nabla }}_{\theta }{E}_{\theta }(x^{i,k}))\mathrm{.}$
3. Update $\theta \leftarrow \theta -\eta {\widehat{g}}_k$.

When ${\mathrm{\nabla }}_{\theta }{E}_{\theta }$ is integrable under $Q_0$ and $Q_{k,\theta }$, the associated population CD-$k$ direction is $g_k(\theta ):=\int {\mathrm{\nabla }}_{\theta }{E}_{\theta }d{Q}_0-\int {\mathrm{\nabla }}_{\theta }{E}_{\theta }d{Q}_{k,\theta }\mathrm{.}$ The procedure makes no claim that the chain has mixed in $k$ steps. The parameter derivative of $Q_{k,\theta }$ is not stopped in the scalar objective's definition; the update above deliberately omits that derivative.

Energy-based density and partition function defines the normalized energy law used as the chain target.

Measure kernel and probability kernel defines the probability transition kernels.

Composition of probability kernels defines their iterated composition.

Kernel composition is well defined and associative establishes that finite compositions remain probability kernels.

The measure with density $f$ relative to $\mu$ defines a measure from a nonnegative density.

Probability measures and probability spaces defines the data and model laws.

The nonnegative Lebesgue integral defines the nonnegative integrals.

Integrable real and complex functions, and their integrals defines integrability of the relative-entropy terms and parameter gradients.