Finite masked-diffusion generative model

For the finite masking state space $S=(V\cup \{m\}{)}^L$ and reverse kernels $P_j$ of the finite-grid sampler, put a point-mass prior at $Z_N=m^L$. Define a decoder $D$ from $S$ to $V^L$ by $D(x\mid z_0)={\mathbf{1}}_{\{x=z_0\}}$ when $z_0\in V^L$, and $D(x\mid z_0)=\mathrm{\mid }V{\mathrm{\mid }}^{-L}$ for every $x\in V^L$ when $z_0\mathrm{\notin }{V}^L$. The finite joint mass of a clean output $x\in V^L$ and latent path $(z_0,\dots ,z_N)\in S^{N+1}$ is $p(x,z_0,\dots ,z_N)={\mathbf{1}}_{\{z_N=m^L\}}({\prod }_{j=1}^N{P}_j(z_j,z_{j-1}))D(x\mid z_0)\mathrm{.}$ Thus reverse step $j$ maps the state at level $j$ to the state at level $j-1$; the decoder is deterministic on clean endpoint states and is explicitly normalized on all other states.

Finite masking diffusion chain: fixes the finite vocabulary, mask, sequence length, grid, survival schedule, and sequence state space.

Finite-grid masked reverse sampler: supplies each reverse kernel $P_j$ and starts at $m^L$.

Measure kernel and probability kernel: a probability kernel supplies normalized transition rows.