Green kernel of a transient chain

For any countable transition matrix $p$, define its extended nonnegative Green kernel by

$G(x,y):={\sum }_{n\ge 0}{p}^{(n)}(x,y)\in [0,+\mathrm{\infty }]\mathrm{.}$

This matrix series is defined without Choice. Under the explicitly assumed Axiom of Choice The Axiom of Choice, for a specified chain law with initial state $x$, the multistep Markov identity gives ${\mathbb{P}}_x(X_n=y)=p^{(n)}(x,y)$ for each $n$ (Chapman-Kolmogorov equations). Applying monotone convergence to the partial sums of $N_y={\sum }_{n\ge 0}{\mathbf{1}}_{\{X_n=y\}}$ then gives

${\mathbb{E}}_x{N}_y={\sum }_{n\ge 0}{\mathbb{P}}_x(X_n=y)=G(x,y)\mathrm{.}$

No finiteness is asserted: $G(x,y)$ may be $+\mathrm{\infty }$, including when a chain starts in a transient state and can enter a recurrent class. The expectation identity uses Hitting, return, and visit times and Monotone convergence for the integral; the matrix definition remains valid for recurrent states as well.