Positive and null recurrence of a state

Let a Markov chain with transition matrix $p$ and a fixed deterministic initial state $x$ be specified, and use ${\mathbb{P}}_x$ for its law. Let

$T_x^{+}=\inf \{n\ge 1:X_n=x\}$

be the first strictly positive return time of Hitting, return, and visit times; it never counts the initial visit at time zero and may equal $+\mathrm{\infty }$. A state $x$ that is recurrent, that is ${\mathbb{P}}_x(T_x^{+}<\mathrm{\infty })=1$ (Recurrent and transient states), is

• positive recurrent when ${\mathbb{E}}_x{T}_x^{+}<+\mathrm{\infty }$, and
• null recurrent when ${\mathbb{E}}_x{T}_x^{+}=+\mathrm{\infty }$.

The expectation is the extended nonnegative integral of the ${\mathbb{N}}_0\cup \{+\mathrm{\infty }\}$-valued random variable $T_x^{+}$, so it always exists in $[0,+\mathrm{\infty }]$ and the two cases are exhaustive and mutually exclusive for a recurrent state. A state that is transient is in neither subclass, since its return probability is strictly less than one. A finite mean forces almost-sure finiteness: if a nonnegative integer-valued random variable is infinite with positive probability, its extended expectation is $+\mathrm{\infty }$. The classification is stated for a fixed specified law ${\mathbb{P}}_x$; no simultaneous selection of laws for all states is asserted here.