Convergence and integrable envelope for finite-past information

Choose the null-set versions allowed by [F2] to be zero where $p_k(X_0)=0$, so each $g_k$ is nonnegative without changing any almost-sure claim. For each $a\in \mathcal{A}$, $p_k(a)=E[1_{\{X_0=a\}}\mid {\mathcal{F}}_k]$ is a martingale in $k$ for ${\mathcal{F}}_k=\sigma (X_{-k},\dots ,X_{-1})$, because the finite-atom conditional averages satisfy the tower identity when the partition is refined. It is bounded between zero and one.

For rationals $0\le u<v\le 1$, the buy-at-$u$, sell-at-$v$ strategy applied to this bounded martingale has expected gain zero; each completed upcrossing earns at least $v-u$, and an unfinished final holding loses at most one. Therefore the expected number of completed upcrossings by time $N$ is at most $(v-u{)}^{-1}$. By [F7], the total upcrossing count has finite expectation and is finite almost surely. Intersect over the countably many rational pairs and the finite alphabet: each $p_k(a)$ then converges to some $p_{\mathrm{\infty }}(a)\in [0,1]$.

For every finite-past cylinder $B\in {\mathcal{F}}_m$, the tower identity gives ${\int }_B{p}_k(a)dP=P(B\cap \{X_0=a\})$ for all $k\ge m$. Since $0\le p_k(a)\le 1$ and $p_k(a)\to p_{\mathrm{\infty }}(a)$ almost surely, [F6] gives the same identity with $p_{\mathrm{\infty }}(a)$. For fixed $a$, the finite measures $B\mapsto {\int }_B{p}_{\mathrm{\infty }}(a)dP$ and $B\mapsto P(B\cap \{X_0=a\})$ agree on the algebra of finite-past cylinders. Their agreement class is a monotone class by continuity of finite measures, so [F5] extends the identity to every $B\in {\mathcal{F}}^{-}$. Thus $p_{\mathrm{\infty }}(a)$ is a conditional probability of $X_0=a$ given the full past.

The event $\{p_{\mathrm{\infty }}(a)=0\}$ belongs to ${\mathcal{F}}^{-}$. The conditional-probability identity from step 3.1 gives $P(X_0=a,p_{\mathrm{\infty }}(a)=0)=\int 1_{\{p_{\mathrm{\infty }}(a)=0\}}{p}_{\mathrm{\infty }}(a)dP=0.$ Summing over the finite alphabet shows $p_{\mathrm{\infty }}(X_0)>0$ almost surely, so $g_k\to g_{\mathrm{\infty }}:=-{\log }_2{p}_{\mathrm{\infty }}(X_0)$ almost surely.

For $t\ge 0$ and $a\in \mathcal{A}$, let ${\tau }_a$ be the first $k$ with $p_k(a)<2^{-t}$. On each event $\{{\tau }_a=k\}\in {\mathcal{F}}_k$, the tower identity gives $P(X_0=a,{\tau }_a=k)={\int }_{\{{\tau }_a=k\}}{p}_k(a)dP\le 2^{-t}P({\tau }_a=k)\mathrm{.}$ Summing over $k$ and then $a$ yields $P(G>t)\le \mathrm{\mid }\mathcal{A}\mathrm{\mid }{2}^{-t}$ for $G={\sup }_k{g}_k$. For each integer $M\ge 1$, $G\wedge M\le 1+{\sum }_{j=1}^{M-1}{1}_{\{G>j\}},$ so [F7] and the tail bound give $\int GdP={\lim }_M\int (G\wedge M)dP\le 1+\mathrm{\mid }\mathcal{A}\mathrm{\mid }{\sum }_{j\ge 1}{2}^{-j}<\mathrm{\infty }$. Since $g_k\le G$ and $g_{\mathrm{\infty }}\le G$ almost surely, [F6] also gives $\int g_{k}dP\to \int g_{\mathrm{\infty }}dP$.

By stationarity [F1] and the chain rule [F3], $H_n={\sum }_{k=0}^{n-1}H(X_k\mid X_0^{k-1})={\sum }_{k=0}^{n-1}H(X_0\mid X_{-k}^{-1})={\sum }_{k=0}^{n-1}\int g_{k}dP\mathrm{.}$ Cesàro convergence and step 5.1 give ${\lim }_n{H}_n\mathrm{/}n=\int g_{\mathrm{\infty }}dP$; [F4] identifies this limit as $h$. This also covers zero-probability transitions, which are never observed.