Communicating classes in a four-state chain

The four displayed rows are nonnegative and each sums to one. Because $P$ is a transition matrix, every unlisted entry in each row must therefore be zero; the matrix is fully specified as displayed.

By induction using [F4], for every $n\ge 0$ the row $p^{(n)}(0,\cdot )$ is concentrated at $0$, while the rows from $1$ and $2$ alternate deterministically between those two states. Thus $0$ reaches neither $1,2,3$, and $1,2$ reach neither $0$ nor $3$. Since $p(1,2)=p(2,1)=1$, states $1,2$ communicate. State $3$ communicates with itself, and it reaches $0$, but $0$ cannot reach $3$; its rows also show it reaches neither $1$ nor $2$. Hence the communication classes are exactly $\{0\}$, $\{1,2\}$, and $\{3\}$.

From state $0$ the chain stays at $0$, so $T_0^{+}=1$ almost surely. From states $1$ and $2$ it alternates deterministically, so $T_1^{+}=T_2^{+}=2$ almost surely. In all three cases the positive return probability is one, so $0,1,2$ are recurrent by [F6].

From state $3$, the first step is a return to $3$ with probability $p(3,3)=1\mathrm{/}2$. With the remaining probability $1\mathrm{/}2$ the chain moves to $0$ and then stays there forever, so there is no later return to $3$. Therefore ${\mathbb{P}}_3(T_3^{+}<\mathrm{\infty })=1\mathrm{/}2<1$, and state $3$ is transient by [F7].

The example fixes a four-state space, so the empty-space and one-state cases do not arise. Zero entries are forced by row normalization and are used in the access calculation; the deterministic rows and absorbing state are covered directly. Accessibility includes the zero-step identity, whereas $T_x^{+}$ starts at time one. All calculations are finite and choice-free. This example gives a state classification, not an iff theorem.