Positive recurrence and stationary probability for irreducible countable chains

For every $n\ge 0$ one has $\pi p^{(n)}=\pi$ whenever $\pi$ is an invariant probability, where $(\pi p^{(n)})(y):={\sum }_{x\in E}\pi (x)p^{(n)}(x,y)$: the case $n=0$ is $p^{(0)}(x,y)={\mathbf{1}}_{\{x=y\}}$, and if $\pi p^{(n)}=\pi$, then $(\pi p^{(n+1)})(y)={\sum }_x\pi (x){\sum }_z{p}^{(n)}(x,z)p(z,y)={\sum }_z({\sum }_x\pi (x)p^{(n)}(x,z))p(z,y)={\sum }_z\pi (z)p(z,y)=\pi (y)$ for every $y$, using [F2] with $m=n$, $n=1$ and the interchange of the two nonnegative series in [F6].

Conversely, assume some state $b$ is positive recurrent. Then ${\mathbb{P}}_b(T_b^{+}<\mathrm{\infty })=1$ and $0<{\mathbb{E}}_b{T}_b^{+}<\mathrm{\infty }$; by [F4] the measure ${\mu }_b$ satisfies ${\mu }_{b}p={\mu }_b$, and ${\mu }_b\ge 0$ with $1={\mu }_b(b)\le {\sum }_y{\mu }_b(y)={\mathbb{E}}_b{T}_b^{+}<\mathrm{\infty }$; hence ${\pi }_{\ast }(y):={\mu }_b(y)\mathrm{/}{\mathbb{E}}_b{T}_b^{+}$ defines a probability vector with ${\pi }_{\ast }p={\pi }_{\ast }$, i.e. an invariant probability by [F5].

Assume there is an invariant probability $\pi$. Then $\pi (b)>0$ for every $b\in E$: by [F1] irreducibility gives $n$ with $p^{(n)}(x,b)>0$ for an arbitrary fixed $x$, and step 1.1 gives $\pi (b)={\sum }_{x^{\mathrm{\prime }}\in E}\pi (x^{\mathrm{\prime }})p^{(n)}(x^{\mathrm{\prime }},b)\ge \pi (x)p^{(n)}(x,b)$, so if $\pi (b)=0$ then $\pi (x)=0$ for every $x$, contradicting ${\sum }_x\pi (x)=1$.

With $\pi$ invariant, define $\nu (y):=\pi (y)\mathrm{/}\pi (b)$ for $y\in E$; this is well defined and finite by step 2.1, $\nu \ge 0$, $\nu (b)=1$, and for $y\mathrm{\ne }b$ the invariance identity [F5] gives $(\nu p)(y)={\sum }_x\nu (x)p(x,y)=\frac{1}{\pi (b)}{\sum }_x\pi (x)p(x,y)=\pi (y)\mathrm{/}\pi (b)=\nu (y)$.

The pointwise minimality of [F4] applied to $\nu$ yields ${\mu }_b(y)\le \nu (y)$ for every $y$; summing and using ${\sum }_y{\mu }_b(y)={\mathbb{E}}_b{T}_b^{+}$ from [F4] gives ${\mathbb{E}}_b{T}_b^{+}\le {\sum }_y\nu (y)=1\mathrm{/}\pi (b)<+\mathrm{\infty }$, so $b$ is recurrent with finite expected return time, i.e. positive recurrent by [F3]. Since $b$ was arbitrary, every state is positive recurrent.

The three statements are equivalent: every state positive recurrent implies some state positive recurrent because $E\mathrm{\ne }\mathrm{\varnothing }$; some state positive recurrent implies the existence of an invariant probability by step 1.2; and the existence of an invariant probability implies every state positive recurrent by steps 2.1–4.1. In the construction of step 1.2, ${\pi }_{\ast }(y)={\mu }_b(y)\mathrm{/}{\mathbb{E}}_b{T}_b^{+}$ and ${\pi }_{\ast }(b)={\mu }_b(b)\mathrm{/}{\mathbb{E}}_b{T}_b^{+}=1\mathrm{/}{\mathbb{E}}_b{T}_b^{+}$, which are the two displayed formulas of the statement, while the bound ${\mathbb{E}}_b{T}_b^{+}\le 1\mathrm{/}\pi (b)$ for an invariant $\pi$ is step 4.1.

Boundary and axiom cases: if $E$ is a singleton then $p(1,1)=1$, every state is positive recurrent with $T_b^{+}=1$, and $\pi ={\delta }_b$ is the invariant probability, consistent with all three clauses; no state is transient here, so the alternatives of [F3] are exhaustive; the equivalence is proved in both directions through steps 4.1 and 1.2, not assumed; the arguments never subtract infinite quantities, since all sums of occupation masses are nonnegative and are shown finite only after the minimality bound; and AC [A1] is used exactly through [F4], the published chain-law and return-cycle supplier, whose statement assumes AC, while the remaining steps are nonnegative matrix algebra.