Discrete strong Markov property

Since $h$ is measurable, $(h(X_n))$ is adapted. Applying [F3] with value [F1, F3] zero at infinity shows that $R_h$ is ${\mathcal{F}}_{\tau }$-measurable. Moreover $\mathrm{\mid }{Z}_H\mathrm{\mid },\mathrm{\mid }{R}_h\mathrm{\mid }\le \parallel H{\parallel }_{\mathrm{\infty }}$, so both variables are integrable. This also covers $H=0$, constant $H=1$, and the event $\{\tau =\mathrm{\infty }\}$, where both variables vanish by definition.

Fix $A\in {\mathcal{F}}_{\tau }$. For every finite $n$, [F2] and [F1] give [F1, F2, F4, step 1.1] $\begin{array}{rl}{\displaystyle }& {\displaystyle \mathbb{E}[1_{A\cap \{\tau =n\}}H(X_n,X_{n+1},\dots )]}\\ {\displaystyle }& {\displaystyle =\mathbb{E}[1_{A\cap \{\tau =n\}}h(X_n)]\mathrm{.}}\end{array}$ Sum from $n=0$ to $N$. The partial sums on either side are bounded in absolute value by $\parallel H{\parallel }_{\mathrm{\infty }}$ and converge pointwise to $1_A{Z}_H$ and $1_A{R}_h$. By [F4], letting $N\to \mathrm{\infty }$ gives $\mathbb{E}[1_A{Z}_H]=\mathbb{E}[1_A{R}_h]$. Together with step 1.1 this is the defining event test for the displayed conditional expectation. Empty $A$, full $A$, $\tau =0$, and an almost-surely infinite $\tau$ require no separate argument.

If $\mathbb{P}(\tau <\mathrm{\infty })=1$, the exceptional infinity event is null, so [step 2.1] $Z_H=H(X_{\tau },X_{\tau +1},\dots )$ and $R_h=h(X_{\tau })$ almost surely for any arbitrary values assigned there. This proves the finite form. Choice is used only by [F1] and the conditional expectation in the conclusion.