Convergence to stationarity for irreducible aperiodic positive-recurrent chains

Uniqueness of the invariant probability: by [F1] positive recurrence supplies an invariant probability ${\pi }_{\ast }$. Let $\rho$ be any invariant probability and fix an arbitrary $y\in E$. Applying [F10] to each of ${\pi }_{\ast }$ and $\rho$ gives ${\pi }_{\ast }(y)=1\mathrm{/}{\mathbb{E}}_y{T}_y^{+}=\rho (y)$. Since this holds for every $y$, $\rho ={\pi }_{\ast }$ pointwise, so the invariant probability $\pi$ of the statement is unique.

Define the product kernel $Q$ on $E\times E$ by $Q((u,v),(u^{\mathrm{\prime }},v^{\mathrm{\prime }})):=p(u,u^{\mathrm{\prime }})p(v,v^{\mathrm{\prime }})$. Its rows sum to one by [F8] and stochasticity of $p$, so $Q$ is a transition matrix.

For the total variation distance, [F7] gives $\parallel p^{(n)}(x,\cdot )-\pi {\parallel }_{\mathrm{T}\mathrm{V}}=\frac{1}{2}{\sum }_{z\in E}\mathrm{\mid }{p}^{(n)}(x,z)-\pi (z)\mathrm{\mid }$; since both $p^{(n)}(x,\cdot )$ and $\pi$ are probability laws on $E$, $\mathrm{\mid }{a}_z-b_z\mathrm{\mid }=a_z+b_z-2\min (a_z,b_z)$ termwise, so the half-sum equals $1-{\sum }_{z\in E}\min (p^{(n)}(x,z),\pi (z))$.

The product transition probabilities satisfy $Q^{(n)}((u,v),(u^{\mathrm{\prime }},v^{\mathrm{\prime }}))=p^{(n)}(u,u^{\mathrm{\prime }})p^{(n)}(v,v^{\mathrm{\prime }})$: this is true at $n=0$, and [F9] gives the $n+1$ sum over $(a,b)$; the induction hypothesis factors that double nonnegative sum into the two one-coordinate sums by [F8], after which [F9] gives the claimed formula. Thus for states $(u,v)$ and $(u^{\mathrm{\prime }},v^{\mathrm{\prime }})$, choose $n\ge \max \{N_{u,u^{\mathrm{\prime }}},N_{v,v^{\mathrm{\prime }}}\}$ from [F2]; the product formula gives $Q^{(n)}((u,v),(u^{\mathrm{\prime }},v^{\mathrm{\prime }}))>0$, so $Q$ is irreducible.

By [F5], the product kernel $Q$ of step 1.2 has a canonical chain $((X_n,Y_n){)}_{n\ge 0}$ with initial law ${\delta }_x\otimes \pi$, so $X_0=x$ and $\mathcal{L}(Y_0)=\pi$. Marginalizing a $Q$-transition row over the other coordinate gives the corresponding $p$-row, so each coordinate is a $p$-chain; because $Y$ starts with invariant law $\pi$, [F6] gives $\mathcal{L}(Y_n)=\pi$ for all $n$.

The product measure $\pi \otimes \pi$ is invariant for $Q$: $((\pi \otimes \pi )Q)(u^{\mathrm{\prime }},v^{\mathrm{\prime }})={\sum }_{u,v}\pi (u)\pi (v)p(u,u^{\mathrm{\prime }})p(v,v^{\mathrm{\prime }})=({\sum }_u\pi (u)p(u,u^{\mathrm{\prime }}))({\sum }_v\pi (v)p(v,v^{\mathrm{\prime }}))=\pi (u^{\mathrm{\prime }})\pi (v^{\mathrm{\prime }})$, using invariance of $\pi$ and the interchange of nonnegative double sums [F8].

Aperiodicity is used in step 2.1 through [F2]. Without it, the deterministic two-cycle has a point-mass $n$-step law from $0$ and uniform stationary law, so under the sup-over-events convention [F7] the event $\{0\}$ realizes distance $1\mathrm{/}2$ at every $n$.

By steps 2.1–2.2 and the equivalence [F1], the product chain is positive recurrent, hence recurrent, and [F3] applied to its irreducible class gives, for every diagonal state $d_y:=(y,y)$ and every initial state $(u,v)$, ${\mathbb{P}}_{(u,v)}(T_{d_y}<\mathrm{\infty })=1$.

Let $\mathrm{\Delta }:=\{(y,y):y\in E\}$ and $T:=T_{\mathrm{\Delta }}=\inf \{n\ge 0:X_n=Y_n\}$. Then $\mathbb{P}(T<\mathrm{\infty })=1$ under the law of step 2.2: conditionally on $Y_0=y$ one has $T\le T_{(y,y)}$, so $\mathbb{P}(T<\mathrm{\infty })\ge {\mathbb{P}}_{(x,y)}(T_{(y,y)}<\mathrm{\infty })$ for each $y$, and averaging over the law $\pi$ of $Y_0$ with step 3.2 gives $\mathbb{P}(T<\mathrm{\infty })\ge {\sum }_y\pi (y)\cdot 1=1$.

Construct a process $W$: set $W_n:=X_n$ for $n\le T$ and $W_n:=Y_n$ for $n>T$. By the strong Markov property [F4] applied to the product chain at the stopping time $T$, the post-$T$ path given ${\mathcal{F}}_T$ is a product chain started at the diagonal state $(Z,Z)$, $Z:=X_T=Y_T$; hence its second coordinate is a $p$-chain started at $Z$ and measurable in the post-$T$ randomness alone. Concatenating the $X$-path up to $T$ with that second coordinate therefore yields a process with the law of a canonical $p$-chain started at $x$, so $\mathcal{L}(W_n)=p^{(n)}(x,\cdot )$ for every $n$.

Since $W_n=Y_n$ for all $n>T$ and both processes are defined everywhere, $\{W_n\mathrm{\ne }{Y}_n\}\subseteq \{T>n\}$; consequently for each $z\in E$ and each $n$, $\mathrm{\mid }{p}^{(n)}(x,z)-\pi (z)\mathrm{\mid }=\mathrm{\mid }\mathbb{P}(W_n=z)-\mathbb{P}(Y_n=z)\mathrm{\mid }\le \mathbb{P}(T>n)$, using $\mathcal{L}(Y_n)=\pi$ from step 2.2. Hence $p^{(n)}(x,z)\to \pi (z)$ for every $z$, since $\mathbb{P}(T<\mathrm{\infty })=1$ by step 4.1.

The minimum sum converges to $1$: given $\epsilon >0$, countable additivity of $\pi$ supplies a finite $F\subseteq E$ with $\pi (F)>1-\epsilon$; by step 6.1, $\min (p^{(n)}(x,z),\pi (z))\to \pi (z)$ for each of the finitely many $z\in F$, so ${\mathrm{lim\hspace{0.17em}inf}}_n{\sum }_{z\in E}\min (p^{(n)}(x,z),\pi (z))\ge {\sum }_{z\in F}\pi (z)>1-\epsilon$; letting $\epsilon \downarrow 0$ and using ${\sum }_z\min (\cdot ,\cdot )\le {\sum }_z\pi (z)=1$ gives convergence of the full sum to $1$.

The pointwise comparison in step 6.1 handles both signs of the difference, so no separate converse case is needed.

Combining steps 1.3 and 7.1, $\parallel p^{(n)}(x,\cdot )-\pi {\parallel }_{\mathrm{T}\mathrm{V}}\to 0$ for the arbitrary starting state $x$, which is the assertion.

Step 7.1 takes a finite high-mass subset before passing to the limit; it does not interchange a limit with an infinite sum.

If $E$ is a singleton, both laws coincide for every $n$; the same argument applies to any starting state because $x$ entered only through the initial law ${\delta }_x\otimes \pi$.

AC [A1] is used exactly at the AC-qualified supplier applications in steps 1.1, 2.2, 3.2, and 5.1; the finite comparison and truncation arguments are choice-free.