Equivalent criteria for recurrence and transience

For each $k\ge 1$, pathwise $\{N_x\ge k\}=\{R_{k-1}<\mathrm{\infty }\}$. Since $X_0=x$ [F1], the first visit is already counted, and at least $k$ visits are exactly $k-1$ further finite returns. In particular, both events are certain for $k=1$ since $R_0=0$. Also $\{N_x=\mathrm{\infty }\}={\bigcap }_{k\ge 1}\{N_x\ge k\}$.

The finite partial visit counts $V_m:={\sum }_{n=0}^m{\mathbf{1}}_{\{X_n=x\}}$ are nonnegative measurable by [F9] and increase pointwise to $N_x$ [F2]. Monotone convergence [F10] therefore gives, with extended values allowed, ${\mathbb{E}}_x{N}_x={\lim }_{m\to \mathrm{\infty }}{\sum }_{n=0}^m{\mathbb{P}}_x(X_n=x)={\sum }_{n=0}^{\mathrm{\infty }}{p}^{(n)}(x,x)$, where the last equality uses the finite-dimensional law [F7] under AC [A1]. The $n=0$ term is $1$ on each side by [F1] and [F6]; no initial visit is lost.

Define the bounded future-path functional $H_x(\omega )={\mathbf{1}}_{\{\mathrm{\exists }m\ge 1:{\omega }_m=x\}}$. Its event is a countable union of coordinate-cylinder events, hence measurable by [F9], and ${\mathbb{E}}_x{H}_x=r_x$ by the definition of $T_x^{+}$ [F3]. On $\{R_j<\mathrm{\infty }\}$, $Z_{j,H_x}={\mathbf{1}}_{\{R_{j+1}<\mathrm{\infty }\}}$; on $\{R_j=\mathrm{\infty }\}$ it is zero. Applying the AC-based strong-Markov identity [F8] and taking expectations gives ${\mathbb{P}}_x(R_{j+1}<\mathrm{\infty })=r_x{\mathbb{P}}_x(R_j<\mathrm{\infty })$. Starting from ${\mathbb{P}}_x(R_0<\mathrm{\infty })=1$, induction yields ${\mathbb{P}}_x(R_j<\mathrm{\infty })=r_x^j$. By step 1.1, ${\mathbb{P}}_x(N_x\ge k)=r_x^{k-1}$ for every $k\ge 1$.

The events $\{N_x\ge k\}$ decrease to $\{N_x=\mathrm{\infty }\}$. Continuity from above [F11] and step 2.1 give ${\mathbb{P}}_x(N_x=\mathrm{\infty })={\lim }_{k\to \mathrm{\infty }}{r}_x^{k-1}$. If $x$ is recurrent, [F5] gives $r_x=1$, so this limit is $1$. If $x$ is transient, [F5] gives $0\le r_x<1$, and [F12] makes the limit $0$. These two cases exhaust all states, so $x$ is recurrent if and only if ${\mathbb{P}}_x(N_x=\mathrm{\infty })=1$.

For $M\ge 1$, let $W_M:={\sum }_{k=1}^M{\mathbf{1}}_{\{N_x\ge k\}}$. These are nonnegative measurable variables, increase pointwise to $N_x$, and [F10] together with step 2.1 gives ${\mathbb{E}}_x{N}_x={\sum }_{k=1}^{\mathrm{\infty }}{\mathbb{P}}_x(N_x\ge k)={\sum }_{j=0}^{\mathrm{\infty }}{r}_x^j$. If $x$ is recurrent then [F5] gives $r_x=1$ and this sum is $+\mathrm{\infty }$. If $x$ is transient then [F5] gives $r_x<1$ and [F13] gives ${\mathbb{E}}_x{N}_x=1\mathrm{/}(1-r_x)<\mathrm{\infty }$. In view of step 1.2, the Green series diverges exactly in the recurrent case. This proves both directions of the recurrence/Green-series equivalence without subtracting extended values.

In the transient case, for every $k\ge 1$ the nested tail events satisfy ${\mathbb{P}}_x(N_x=k)={\mathbb{P}}_x(N_x\ge k)-{\mathbb{P}}_x(N_x\ge k+1)=(1-r_x)r_x^{k-1}$ by step 2.1 and finite subtraction of probabilities in $[0,1]$. Since ${\mathbb{P}}_x(N_x=\mathrm{\infty })=0$ by step 3.1, these masses account for all outcomes; their sum is $(1-r_x){\sum }_{j\ge 0}{r}_x^j=1$ by [F13]. When $r_x=0$, the convention $0^0=1$ [F14] gives ${\mathbb{P}}_x(N_x=1)=1$ and ${\mathbb{P}}_x(N_x=k)=0$ for $k>1$, as expected when no positive return occurs. The mean formula is the transient case of step 3.2.

If $E=\mathrm{\varnothing }$, there is no $x$ and the theorem is vacuous. For a one-state absorbing chain, $r_x=1$, $N_x=\mathrm{\infty }$ almost surely, and $p^{(n)}(x,x)=1$ for every $n$, agreeing with both recurrence criteria. If a deterministic chain started at $x$ leaves and never returns, then $r_x=0$, $N_x=1$ almost surely, and $p^{(n)}(x,x)=0$ for $n\ge 1$, agreeing with the transient formulas. If instead a deterministic cycle returns after a fixed positive period $d$, then $r_x=1$ and $p^{(md)}(x,x)=1$ for all $m\ge 0$, so both the infinite-visit probability and Green series are infinite as asserted. The $n=0$ Green term and $k=1$ tail were treated explicitly in steps 1.1 and 1.2–3.2. AC [A1] is used for the specified chain law and the conditional strong-Markov identity [F8]; the return-series arithmetic itself uses no choice. Both stated equivalences have been proved in both directions in steps 3.1 and 3.2.