Markov Kernels and Markov Chains — Examples

For each $x$, $A\mapsto K(x,A)=\nu (A)$ is a probability measure; for each [given] $A$, $x\mapsto \nu (A)$ is constant and measurable. Thus $K$ is a probability kernel, including $\nu (A)=0$ and $1$ and a one-point state space.

Put ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$. By [F1]--[F2], [F1, F2, F3, step 1.1] $\sigma (X_{n+1})$ is independent of ${\mathcal{F}}_n$. Hence, for $B\in {\mathcal{F}}_n$ and $A\in \mathcal{E}$, $\mathbb{E}[1_B{1}_{\{X_{n+1}\in A\}}]=\mathbb{P}(B)\nu (A)=\mathbb{E}[1_{B}K(X_n,A)]\mathrm{.}$ Therefore the constant $K(X_n,A)=\nu (A)$ is a version of the indicated conditional probability. By [F3], equivalently $\mathbb{E}[f(X_{n+1})\mid {\mathcal{F}}_n]=\int fd\nu$ for every bounded measurable $f$. This verifies the claim at $n=0$ and all later times. Choice is used only for those conditional-expectation classes.

For fixed $x$, $K(x,\cdot )={\delta }_{Tx}$ is a probability measure. For fixed [F1] $A$, $K(x,A)=1_{T^{-1}A}(x)$ is measurable. Thus [F1] holds, including empty and full $A$ and a one-point state space.

For $A\in \mathcal{E}$, [F2, step 1.1] $1_{\{X_{n+1}\in A\}}=1_A(T{X}_n)=K(X_n,A)\text{a.s.}$ The last variable is ${\mathcal{F}}_n$-measurable, so it is its own conditional expectation given ${\mathcal{F}}_n$. By [F2], $X$ is a $K$-chain. This also covers $n=0$, constant maps, fixed points, and deterministic cycles. Choice is used only for the conditional-expectation class in [F2]; the kernel construction is choice-free.

The natural past $\sigma (X_0,\dots ,X_n)$ equals [F1, F2] $\sigma ({\xi }_1,\dots ,{\xi }_n)$ because $x_0$ is fixed and ${\xi }_k=X_k-X_{k-1}$. By [F1], ${\xi }_{n+1}$ is independent of this past. Thus for every $A\subseteq \mathbb{Z}$, $\mathbb{P}(X_{n+1}\in A\mid {\mathcal{F}}_n)=\frac{1}{2}{1}_A(X_n+1)+\frac{1}{2}{1}_A(X_n-1)=p(X_n,A)\mathrm{.}$ This includes $A=\mathrm{\varnothing }$, $A=\mathbb{Z}$, and $n=0$, and verifies [F2]. Choice is used only for the displayed conditional expectation.

Only $y=x+1$ and $y=x-1$ contribute to [F3], so [F3, step 1.1] $\begin{array}{rl}{\displaystyle Lf(x)}& {\displaystyle =\frac{1}{2}(f(x+1)-f(x))+\frac{1}{2}(f(x-1)-f(x))}\\ {\displaystyle }& {\displaystyle =\frac{1}{2}\{f(x+1)-2f(x)+f(x-1)\}\mathrm{.}}\end{array}$ The sum is finite, and constants give $Lf=0$.

For $0<x<N$, the only row masses are $p$ and $q$, which are nonnegative and [F1, F2] sum to one. At $0$ and $N$ the rows are the corresponding Dirac masses. Thus these rows are exactly [F1] applied to any base kernel having the displayed interior transitions, and [F2] verifies the kernel. When $N=1$ there are no interior rows; when $p=0$ or $p=1$ the interior motion is deterministic.

Define [given] ${\tau }_D=\inf \{n\ge 0:X_n\in \{0,N\}\}\mathrm{.}$ Then $\{{\tau }_D\le n\}={\bigcup }_{k=0}^n\{X_k\in D\}\in {\mathcal{F}}_n$, so it is a hitting time. If $X_0\in D$, then ${\tau }_D=0$; otherwise it may be infinite in the general eventwise formulation.

By [F3], on $\{{\tau }_D<\mathrm{\infty }\}$ the conditional future is the chain started [F3, step 1.1, step 1.2] from $X_{{\tau }_D}$. Step 1.1 gives $P(X_{{\tau }_D},X_{{\tau }_D})=1$, so every subsequent coordinate equals the same boundary state. Constants zero and one give respectively zero and the finite-hit event in the eventwise formula. Choice is used only through [F3].

Let $\gamma =N(0,1)$. For Borel $A$, [F1, F3] $K(x,A)=\int 1_A(ax+\sigma z)\gamma (dz)\mathrm{.}$ For fixed $x$ this is the affine pushforward in [F1], hence a probability measure. The integrand is Borel on ${\mathbb{R}}^2$, so [F3], applied to the constant kernel $x\mapsto \gamma$, makes $x\mapsto K(x,A)$ measurable. Thus $K$ is a probability kernel. For $\sigma =0$ it is the deterministic kernel ${\delta }_{ax}$; empty/full $A$ give zero/one.

The recursion makes [F2, F3, F4, F5, step 1.1] ${\mathcal{F}}_n^X\subseteq \sigma (X_0,Z_1,\dots ,Z_n)$. By [F2], $Z_{n+1}$ is independent of this larger past and hence of ${\mathcal{F}}_n^X$. For a bounded Borel $f$, set ${\varphi }_f(x)=\int f(ax+\sigma z)\gamma (dz),$ which is measurable by [F3]. For $B\in {\mathcal{F}}_n^X$ and Borel $A,C$, independence applied to $B\cap \{X_n\in A\}$ gives $\mathbb{P}(B,X_n\in A,Z_{n+1}\in C)={\int }_B{1}_A(X_n)\gamma (C)d\mathbb{P}\mathrm{.}$ A pi--lambda argument [F4] extends this from rectangles $A\times C$ to every Borel subset of ${\mathbb{R}}^2$, and bounded simple approximation extends it to the function $(x,z)\mapsto f(ax+\sigma z)$. Therefore $\mathbb{E}[f(a{X}_n+\sigma Z_{n+1})\mid {\mathcal{F}}_n^X]={\varphi }_f(X_n)\mathrm{.}$ Since the left side is $\mathbb{E}[f(X_{n+1})\mid {\mathcal{F}}_n^X]$ and ${\varphi }_f=Kf$, [F5] proves the Markov claim. The calculation includes $a=0$, $\sigma =0$, $f=0,1$, and $n=0$. Choice is used in [F1] and in the conditional expectations.

Suppose first that $S=\{s_1,\dots ,s_d\}$ with $d\ge 1$. For each row put [given] $c_0(x)=0,c_j(x)={\sum }_{k=1}^{j}p(x,s_k),1\le j\le d\mathrm{.}$ Then $c_d(x)=1$. Define $F(x,u)=s_j\text{when}\{\begin{array}{ll}{c}_{j-1}(x)\le u<c_j(x),& j<d,\\ c_{d-1}(x)\le u\le 1,& j=d\mathrm{.}\end{array}$ These intervals partition $[0,1]$ with a fixed endpoint convention, even when some row entries vanish. Since $S$ is finite, every inverse image is a finite union of measurable slices, so $F$ is measurable.

Uniform interval lengths give [step 1.1] $\mathbb{P}(F(x,U)=s_j)=c_j(x)-c_{j-1}(x)=p(x,s_j)\mathrm{.}$ The possible singleton endpoint at $1$ has probability zero, so the last closed endpoint does not change this calculation. It covers row probabilities zero and one and the one-state case $d=1$.

Let ${\mathcal{H}}_n=\sigma (X_0,U_1,\dots ,U_n)$; then $X_n$ is [F1, F2, step 1.1, step 2.1] ${\mathcal{H}}_n$-measurable and [F1] makes $U_{n+1}$ independent of ${\mathcal{H}}_n$. For each $s_j$, $1_{\{F(X_n,U_{n+1})=s_j\}}={\sum }_{i=1}^d{1}_{\{X_n=s_i\}}{1}_{\{U_{n+1}\in I_{ij}\}},$ where $I_{ij}$ is the row interval from step 1.1. Conditioning term by term and using step 2.1 gives $p(X_n,s_j)$. Summing over $s_j\in A$ proves the transition identity for every $A\subseteq S$, hence [F2] gives the $p$-chain. Empty/full $A$ and time zero are included. Choice is used only for conditional expectations and, if a canonical realization is requested, by its path-law supplier.

Together with steps 1.1--3.1, consider $S=\mathrm{\varnothing }$: the unique map [given, step 1.1, step 2.1, step 3.1] $\mathrm{\varnothing }\times [0,1]\to \mathrm{\varnothing }$ satisfies the row-law assertion vacuously, but no probability initial law—and hence no chain—exists on $S$.

Let $(X_n)$ be IID with [F1] $\mathbb{P}(X_n=0)=\mathbb{P}(X_n=1)=1\mathrm{/}2$. By [F1], it is Markov with $K(x,\{0\})=K(x,\{1\})=\frac{1}{2}$ for both $x$. Independence gives $\mathbb{P}(X_0=X_1)=\mathbb{P}(0,0)+\mathbb{P}(1,1)=\frac{1}{4}+\frac{1}{4}=\frac{1}{2}\mathrm{.}$

Let $B$ be one fair bit and put $Y_n=B$ for every $n$. Then each $Y_n$ is [F2] again fair, while [F2] makes $Y$ Markov with identity kernel $J$. Here $\mathbb{P}(Y_0=Y_1)=1.$ Moreover $J(0,\{0\})=1\mathrm{\ne }1\mathrm{/}2=K(0,\{0\})$, so the kernels differ.

Thus $\mathcal{L}(X_n)=\mathcal{L}(Y_n)$ for every $n$, including $n=0$, [step 1.1, step 1.2] but their displayed two-time event probabilities are $1\mathrm{/}2$ and $1$. This is a concrete failed conclusion: one-time marginals do not determine even a two-time law, much less the kernel or path law. The state space is nonempty and finite; probabilities zero and one appear explicitly. Choice is inherited only from the Markov-chain interfaces used in [F1]--[F2].

Let $(X_n)$ be IID fair bits and [F1] ${\mathcal{F}}_n^X=\sigma (X_0,\dots ,X_n)$. By [F1], $X$ is a Markov chain for this filtration with $K(x,\{1\})=1\mathrm{/}2$.

Define a larger filtration by [F2] ${\mathcal{G}}_0=\sigma (X_0,X_1),{\mathcal{G}}_n=\sigma (X_0,\dots ,X_n,X_1)(n\ge 1)\mathrm{.}$ Then ${\mathcal{G}}_0={\mathcal{G}}_1$ and ${\mathcal{G}}_n={\mathcal{F}}_n^X$ for $n\ge 1$, so $({\mathcal{G}}_n)$ is increasing; it contains ${\mathcal{F}}_n^X$ at every time, and $X$ is adapted. Thus it meets the structural requirements in [F2].

Since $X_1$ is ${\mathcal{G}}_0$-measurable, [F2, step 1.1, step 1.2] $\mathbb{P}(X_1=1\mid {\mathcal{G}}_0)=1_{\{X_1=1\}}$ almost surely. This differs from $K(X_0,\{1\})=1\mathrm{/}2$ on both positive-probability events $\{X_1=0\}$ and $\{X_1=1\}$. Hence the time-zero identity in [F2] fails for the larger filtration, although it holds naturally. Empty/full target events still give zero/one and do not witness failure. Choice is used only by the conditional-probability classes.

If the displayed process were a homogeneous $K$-chain, its transition from [F1] $X_0=0$ to $X_1=0$ would force $K(0,\{1\})=\mathbb{P}(X_1=1\mid {\mathcal{F}}_0)=0.$ But its next transition from the same state $X_1=0$ to $X_2=1$ would force $K(0,\{1\})=\mathbb{P}(X_2=1\mid {\mathcal{F}}_1)=1,$ a contradiction. The witness uses the same state twice, so changing only the row at state $1$ cannot repair it.

Put $\tilde{S}={\mathbb{N}}_0\times \{0,1\}$ with its power set and let [F1] $b_0=b_1=0$, $b_n=1$ for $n\ge 2$. Define $T(n,x)=(n+1,b_{n+1}),\tilde{K}((n,x),A)=1_A(T(n,x))\mathrm{.}$ Every section is a Dirac probability and every evaluation is measurable. For $Y_n=(n,X_n)$ one has $Y_{n+1}=T(Y_n)$, so its conditional transition is the same kernel $\tilde{K}$ at every time. Hence [F1] now holds on the enlarged state space.

The failed conclusion is therefore specifically the existence of one [step 1.1, step 1.2] homogeneous kernel on the unaugmented state space, not the Markov nature of the time-augmented evolution. The probabilities zero and one, times zero/one/two, and both state endpoints are explicit. Choice is used only to phrase the conditional-probability identities in [F1].