Superharmonic majorants bound exit costs

Fix $x\in E$ and a finite-valued nonnegative $g$. Exhaust finite $E$ by its finite initial subsets, or, for countably infinite $E$, fix an enumeration $e_0,e_1,\dots$ and put $F_j=\{e_0,\dots ,e_{j-1}\}$. Each $g{\mathbf{1}}_{F_j}$ is a finite-range simple function, and the simple-integral formula plus $p(x,y)=K(x,\{y\})$ gives ${\int }_{E}g(y){\mathbf{1}}_{F_j}(y)K(x,dy)={\sum }_{y\in F_j}p(x,y)g(y)$, with zero weights omitted from the row action. These functions increase to $g$, so monotone convergence and the definition of the nonnegative row series give $Kg(x):={\int }_{E}g(y)K(x,dy)=Pg(x)$, also for finite $E$.

For each finite $n$, define $M_n:=\psi (X_{n\wedge T})+{\sum }_{m<n\wedge T}c(X_m)$. By [F1], this is a finite sum of measurable nonnegative terms, equivalently using ${\sum }_{m<n}{\mathbf{1}}_{\{T>m\}}c(X_m)$, so $M_n$ is measurable and finite pathwise. On $\{T\le n\}$, $M_{n+1}=M_n$; on $S_n:=\{T>n\}\in {\mathcal{F}}_n$, one has $X_n\in D$, $M_n=\psi (X_n)+{\sum }_{m<n}c(X_m)$, and $M_{n+1}=\psi (X_{n+1})+{\sum }_{m\le n}c(X_m)$.

For $N\ge 1$, let ${\psi }_N=\min (\psi ,N)$. It is bounded and measurable, so [F6] and step 1.1 give $\mathbb{E}[{\psi }_N(X_{n+1})\mid {\mathcal{F}}_n]=P{\psi }_N(X_n)$ almost surely. As $N\uparrow \mathrm{\infty }$, both ${\psi }_N(X_{n+1})$ and $P{\psi }_N(X_n)$ increase to their untruncated values: for the row action, its supremum over $N$ and over finite row partial sums commute, and each finite partial sum converges termwise. Conditional monotone convergence [F7] therefore yields $\mathbb{E}[\psi (X_{n+1})\mid {\mathcal{F}}_n]=P\psi (X_n)$ almost surely, with the right side finite at every state by hypothesis.

Fix $x\in E$ under ${\mathbb{P}}_x$. Since $M_0=\psi (x)$, suppose inductively that ${\mathbb{E}}_x{M}_n\le \psi (x)$, which makes $M_n$ integrable. Integrating the conditional identity from step 2.1 over $S_n$ by the defining event-integral property [F7] gives ${\mathbb{E}}_x[{\mathbf{1}}_{S_n}\psi (X_{n+1})]={\mathbb{E}}_x[{\mathbf{1}}_{S_n}P\psi (X_n)]$. On $S_n$, $P\psi (X_n)+c(X_n)\le \psi (X_n)$ by the drift hypothesis, and ${\mathbb{E}}_x[{\mathbf{1}}_{S_n}\psi (X_n)]\le {\mathbb{E}}_x{M}_n<\mathrm{\infty }$; thus ${\mathbb{E}}_x[{\mathbf{1}}_{S_n}(\psi (X_{n+1})+c(X_n))]\le {\mathbb{E}}_x[{\mathbf{1}}_{S_n}\psi (X_n)]$. Off $S_n$ the stopped quantities agree, while on $S_n$ their earlier cost sums agree; nonnegative additivity [F12] now yields ${\mathbb{E}}_x{M}_{n+1}\le {\mathbb{E}}_x{M}_n\le \psi (x)$. Induction proves finite stopped expectations without assuming global integrability of $\psi (X_n)$.

Put $Z:=B+{\sum }_{0\le m<T}c(X_m)$, so ${\mathbb{E}}_{x}Z=u(x)$ by [F4]. On $\{T<\mathrm{\infty }\}$, for every $n\ge T$, $M_n=\psi (X_T)+{\sum }_{m<T}c(X_m)\ge f(X_T)+{\sum }_{m<T}c(X_m)=Z$. On $\{T=\mathrm{\infty }\}$, $B=0$ and $M_n\ge {\sum }_{m<n}c(X_m)$, whose partial costs increase to $Z$. Hence $Z\le {\mathrm{lim\hspace{0.17em}inf}}_{n\to \mathrm{\infty }}{M}_n$ pointwise without evaluating $X_{\mathrm{\infty }}$; Fatou [F11] and step 3.1 give $u(x)={\mathbb{E}}_{x}Z\le {\mathbb{E}}_x[{\mathrm{lim\hspace{0.17em}inf}}_n{M}_n]\le {\mathrm{lim\hspace{0.17em}inf}}_n{\mathbb{E}}_x{M}_n\le \psi (x)$. Since $x$ was arbitrary, the claim holds at every state.

If $E=\mathrm{\varnothing }$, there is no state to check. If $D=\mathrm{\varnothing }$, then $T=0$ and the conclusion is $f\le \psi$; if $D=E$, step 4.1 applies with $B=0$. When $f=c=0$, $u=0\le \psi$. For a one-state chain, a boundary start has $T=0$, and if its state is in $D$ then $P\psi =\psi$ forces $c=0$ and $u=0$. Deterministic transitions are covered by the one-step identity. A hit at $T=n+1$ incurs $c(X_n)$ and then the boundary value at $X_{n+1}$. AC [A1] supports the canonical laws, bounded conditional Markov identities, and conditional monotone convergence; the pathwise comparison itself uses no choice. This is a one-way bound, with no iff claim.