Expected exit time solves the Poisson equation

On each path, ${\sum }_{m\ge 0}{\mathbf{1}}_{\{m<T\}}=T$: if $T=t\in {\mathbb{N}}_0$, exactly the indices $m=0,\dots ,t-1$ contribute, while if $T=\mathrm{\infty }$, every index contributes and both sides are $+\mathrm{\infty }$. The sum is empty when $T=0$. Thus the path cost in First-step equations for nonnegative exit costs with boundary payoff $f=0$ and running cost $c=1$ equals $T$, including nonexit paths.

The first-step theorem with $f=0$ and $c=1$ has expected cost $v$ by step 1.1 and [F3], so it gives $v=0$ on $D^c$ and $v(x)=1+Pv(x)$ for $x\in D$. [A1, F3, F4, step 1.1, given] The constant functions are bounded and nonnegative, so the theorem applies; its relation on $D$ is in extended nonnegative arithmetic: $v(x)=1+Pv(x)(x\in D)$

If $v$ is finite at every state, then for each $x\in D$ the first-step relation forces $Pv(x)<\mathrm{\infty }$ and hence $Lv(x)=-1$. [F5, F6, step 2.1, given] Indeed, step 2.1 gives $v(x)=1+Pv(x)<\mathrm{\infty }$, so $Pv(x)<\mathrm{\infty }$. Since $v$ is finite-valued, [F6] defines $Lv(x)$, and the finite-real equation gives $Lv(x)=Pv(x)-v(x)=(v(x)-1)-v(x)=-1.$ Thus the Poisson equation is well-defined at every state of $D$.

If $E$ is finite, $p$ is irreducible, and $D⊊E$, then $A=D^c$ is nonempty. Thus [F7] holds with this target, and [F8] gives ${\mathbb{E}}_x{T}_A<\mathrm{\infty }$ for every $x\in E$. Steps 2.1 and 3.1 give the asserted boundary values and equation. The nonempty-target condition matters: for $D=E$ in a nonempty state space, $T_{\mathrm{\varnothing }}=\mathrm{\infty }$, so the global finite-mean premise fails.

If $E=\mathrm{\varnothing }$, the initial-distribution definition supplies no probability law of an $E$-valued chain; there are no states to check. If $D=\mathrm{\varnothing }$, then $T=0$ from every state, so $v=0$ and the equation on $D$ is vacuous. For a one-state chain $E=\{x\}$, this is the finite irreducible proper-domain case; if instead $D=E$, then $T=\mathrm{\infty }$ and the finite-mean premise fails. As a deterministic-row check, on a finite deterministic cycle and a proper $D$, the nonempty target is reached within at most $\mathrm{\mid }E\mathrm{\mid }-1$ steps; at an interior state $x$ with successor $\sigma (x)$, the first-step identity reads $v(x)=1+v(\sigma (x))$, hence $Lv(x)=-1$. The endpoint $T=0$ is counted by the empty path sum, and for $x\in D$ one has $T\ge 1$, so the unit cost counts precisely the steps strictly before exit. AC is used through the cited first-step and finite hitting-time results; no further choice is made here. This corollary states implications, not an iff.