Renewal decomposition at successive returns

Since $p^{(0)}(x,x)={\mathbf{1}}_{\{x=x\}}$ by [F4], $u_x(0)=1$. The conditional identity [F5] at $m=0$, together with $X_0=x$ [F8] and the definition of $p^{(n)}$ [F4], gives ${\mathbb{P}}_x(X_n=x)=p^{(n)}(x,x)=u_x(n)$.

Every recursively defined $R_k$ is a stopping time: $R_0=0$ is one, and if $R_{k-1}$ is one, then for $n\ge 0$, $\{R_k\le n\}={\bigcup }_{m=0}^{n-1}(\{R_{k-1}=m\}\cap {\bigcup }_{j=m+1}^n\{X_j=x\}),$ with the union empty when $n=0$. For $m\le n$, $\{R_{k-1}=m\}$ is in ${\mathcal{F}}_m$ because it is the difference of the stopping-time events $\{R_{k-1}\le m\}$ and $\{R_{k-1}\le m-1\}$ (with the $m=0$ case immediate). Adaptedness [F9] and the increasing filtration then put every displayed term in ${\mathcal{F}}_n$. This proves the induction using [F1, F2].

Let ${\mathcal{W}}_x$ be the set of excursion words defined in the statement. It is countable because it is a countable union of finite products of the countable set $E$. For any $B\subseteq {\mathcal{W}}_x$, let $H_B$ be the indicator that a path starting at $x$ has a finite first-return word in $B$, and set it to zero if there is no positive return. Its event is a countable union of finite-coordinate cylinder events [F10], so $H_B$ is product-measurable and bounded; write $q_x(B)={\mathbb{E}}_x{H}_B={\mathbb{P}}_x(E_1\in B)$.

Fix $n\ge 1$. The disjoint events $\{X_n=x,T_x^{+}=k\}$, $1\le k\le n$, partition $\{X_n=x\}$, because any path ending at $x$ has a first positive visit by time $n$. By [F5], on $\{T_x^{+}=k\}\in {\mathcal{F}}_k$, ${\mathbb{P}}_x(X_n=x,T_x^{+}=k)={\mathbb{E}}_x[{\mathbf{1}}_{\{T_x^{+}=k\}}{\mathbb{P}}_x(X_n=x\mid {\mathcal{F}}_k)]={\mathbb{E}}_x[{\mathbf{1}}_{\{T_x^{+}=k\}}{p}^{(n-k)}(X_k,x)]=f_x(k)u_x(n-k),$ since $X_k=x$ on that event. Summing these finitely many disjoint contributions and using step 1.1 gives the claimed renewal equation.

By step 1.2, $R_k$ is a stopping time. Apply [F6] to $H_B$ at $R_k$. On $\{R_k<\mathrm{\infty }\}$, $X_{R_k}=x$, and the shifted event $H_B$ is exactly that the next completed excursion word is in $B$. Therefore ${\mathbb{E}}_x[{\mathbf{1}}_{\{R_k<\mathrm{\infty }\}}{\mathbf{1}}_{\{E_{k+1}\in B\}}\mid {\mathcal{F}}_{R_k}]={\mathbf{1}}_{\{R_k<\mathrm{\infty }\}}{q}_x(B),$ where the left side is interpreted as zero when $R_{k+1}=\mathrm{\infty }$. The same strong Markov identity with arbitrary bounded $H$ gives the post-return formula in the statement.

Suppose $x$ is recurrent. Taking $B={\mathcal{W}}_x$ in step 2.2 gives $q_x(B)={\mathbb{P}}_x(T_x^{+}<\mathrm{\infty })=1$ by [F7]. Induction from $R_0=0$ yields ${\mathbb{P}}_x(R_k<\mathrm{\infty })=1$ for every $k$; since there are countably many $k$, all returns are finite simultaneously almost surely.

For $m\ge 1$ and arbitrary $B_1,\dots ,B_m\subseteq {\mathcal{W}}_x$, the event $A={\bigcap }_{j<m}\{E_j\in B_j\}$ belongs to ${\mathcal{F}}_{R_{m-1}}$: on each event $\{R_{m-1}=r\}$ it is determined by $X_0,\dots ,X_r$, so $A\cap \{R_{m-1}\le n\}={\bigcup }_{r=0}^n(A\cap \{R_{m-1}=r\})\in {\mathcal{F}}_n$ by [F3]. Applying the conditional identity in step 2.2 at $R_{m-1}$ and using step 3.1 gives ${\mathbb{P}}_x(E_1\in B_1,\dots ,E_m\in B_m)=q_x(B_m){\mathbb{P}}_x(E_1\in B_1,\dots ,E_{m-1}\in B_{m-1})\mathrm{.}$ Induction in $m$ factors this joint probability as ${\prod }_{j=1}^m{q}_x(B_j)$, proving that the excursion words are iid with common first-excursion law $q_x$.

If $E=\mathrm{\varnothing }$, there is no state $x$ and the theorem is vacuous. If $x$ has no positive return, then $u_x(n)=0$ for every $n\ge 1$ (a visit at positive time would be a return), every $f_x(k)=0$, and the convolution has both sides zero; for $n=0$ the identity is $u_x(0)=1$. At the endpoint $n=1$, the formula is $u_x(1)=f_x(1)u_x(0)$. In a one-state absorbing chain, $f_x(1)=1$, $f_x(k)=0$ for $k>1$, and $u_x(n)=1$, so the equation holds directly. More generally, in a deterministic cycle of length $r$, $f_x(k)={\mathbf{1}}_{\{k=r\}}$ and $u_x(n)={\mathbf{1}}_{\{r\mid n\}}$; if $n<r$ the sum is empty, and if $n\ge r$ the sole possible term is $u_x(n-r)={\mathbf{1}}_{\{r\mid n\}}$, as required. For a transient state the conditional identities of steps 2.2 remain restricted to finite $R_k$; if a return fails, the definition sets later returns to infinity, and no further completed excursion is claimed. AC [A1] is used through [F5] and [F6]; the cylinder measurability and event decomposition use no additional choice. The equation and iid assertion are one-way claims, not biconditionals.