Cesaro convergence for irreducible positive-recurrent chains

Let $f:={\mathbf{1}}_{\{y\}}$. It is bounded with $0\le f\le 1$, and ${\sum }_z\pi (z)\mathrm{\mid }f(z)\mathrm{\mid }=\pi (y)\le 1<\mathrm{\infty }$, so [F1] applies: $\frac{1}{n}{\sum }_{k=0}^{n-1}{\mathbf{1}}_{\{X_k=y\}}\to \pi (y)$ almost surely under ${\mathbb{P}}_x$, for the fixed starting state $x$.

For each $k\ge 0$, ${\mathbb{E}}_x[{\mathbf{1}}_{\{X_k=y\}}]={\mathbb{P}}_x(X_k=y)=K^k(x,\{y\})=p^{(k)}(x,y)$: the second equality is the $m=0$ case of the Chapman–Kolmogorov identity [F3] applied to the indicator of the singleton, and the third is the definition of the $k$-step entries [F4].

Each average $A_n:=\frac{1}{n}{\sum }_{k<n}{\mathbf{1}}_{\{X_k=y\}}$ satisfies $0\le A_n\le 1$ for every $n\ge 1$, and $A_n\to \pi (y)$ almost surely by step 1.1; since ${\mathbb{P}}_x$ is a probability measure, the bounded convergence corollary [F2] applied to the constant limit gives ${\mathbb{E}}_x{A}_n\to \pi (y)$.

By linearity of expectation, ${\mathbb{E}}_x{A}_n=\frac{1}{n}{\sum }_{k=0}^{n-1}{\mathbb{E}}_x[{\mathbf{1}}_{\{X_k=y\}}]=\frac{1}{n}{\sum }_{k=0}^{n-1}{p}^{(k)}(x,y)$; combining with step 2.1 gives $\frac{1}{n}{\sum }_{k=0}^{n-1}{p}^{(k)}(x,y)\to \pi (y)$. Since $x,y$ were arbitrary, the theorem follows.

The average is formed for $n\ge 1$ and includes $p^{(0)}(x,y)={\mathbf{1}}_{\{x=y\}}$. For the periodic two-state alternation it equals $\lceil n\mathrm{/}2\rceil \mathrm{/}n$ or $\lfloor n\mathrm{/}2\rfloor \mathrm{/}n$, according to $x,y$, and tends to $1\mathrm{/}2$; thus no aperiodicity is needed. The bounded indicator satisfies the integrability hypothesis, and AC [A1] is used through [F1] and [F3].