Random-mapping representation for a finite transition matrix

Suppose first that $S=\{s_1,\dots ,s_d\}$ with $d\ge 1$. For each row put [given] $c_0(x)=0,c_j(x)={\sum }_{k=1}^{j}p(x,s_k),1\le j\le d\mathrm{.}$ Then $c_d(x)=1$. Define $F(x,u)=s_j\text{when}\{\begin{array}{ll}{c}_{j-1}(x)\le u<c_j(x),& j<d,\\ c_{d-1}(x)\le u\le 1,& j=d\mathrm{.}\end{array}$ These intervals partition $[0,1]$ with a fixed endpoint convention, even when some row entries vanish. Since $S$ is finite, every inverse image is a finite union of measurable slices, so $F$ is measurable.

Uniform interval lengths give [step 1.1] $\mathbb{P}(F(x,U)=s_j)=c_j(x)-c_{j-1}(x)=p(x,s_j)\mathrm{.}$ The possible singleton endpoint at $1$ has probability zero, so the last closed endpoint does not change this calculation. It covers row probabilities zero and one and the one-state case $d=1$.

Let ${\mathcal{H}}_n=\sigma (X_0,U_1,\dots ,U_n)$; then $X_n$ is [F1, F2, step 1.1, step 2.1] ${\mathcal{H}}_n$-measurable and [F1] makes $U_{n+1}$ independent of ${\mathcal{H}}_n$. For each $s_j$, $1_{\{F(X_n,U_{n+1})=s_j\}}={\sum }_{i=1}^d{1}_{\{X_n=s_i\}}{1}_{\{U_{n+1}\in I_{ij}\}},$ where $I_{ij}$ is the row interval from step 1.1. Conditioning term by term and using step 2.1 gives $p(X_n,s_j)$. Summing over $s_j\in A$ proves the transition identity for every $A\subseteq S$, hence [F2] gives the $p$-chain. Empty/full $A$ and time zero are included. Choice is used only for conditional expectations and, if a canonical realization is requested, by its path-law supplier.

Together with steps 1.1--3.1, consider $S=\mathrm{\varnothing }$: the unique map [given, step 1.1, step 2.1, step 3.1] $\mathrm{\varnothing }\times [0,1]\to \mathrm{\varnothing }$ satisfies the row-law assertion vacuously, but no probability initial law—and hence no chain—exists on $S$.