Markov relative entropy decay and almost-sure time averages

Choose $\mathrm{\Lambda }>{\max }_a{q}_a$ and set $K=I+Q\mathrm{/}\mathrm{\Lambda }$. By [F1], $K$ is stochastic with positive diagonal, and for any distinct $a,b$ a positive-rate path of length $k$ gives $K_{ab}^k>0$; for $a=b$ take $k=0$. By [F2], $P_t=e^{tQ}=e^{-\mathrm{\Lambda }t}{\sum }_{n\ge 0}(\mathrm{\Lambda }t{)}^n{K}^n\mathrm{/}n!$. Every summand is nonnegative and the term for this $k$ is strictly positive, so $P_t(a,b)>0$ for every $t>0$. Hence $p_t(b)={\sum }_a{p}_0(a)P_t(a,b)>0$.

The eigenbasis in [F2] gives, for centered $A_c={\sum }_{j\ge 1}{a}_j{\varphi }_j$, $C_A(s):={\mathbb{E}}_{\pi }[A_c(X_0)A_c(X_s)]=⟨A_c,P_s{A}_c{⟩}_{\pi }={\sum }_{j\ge 1}{a}_j^2{e}^{-{\lambda }_{j}s}$ for $s\ge 0$. Thus $C_A(s)\ge 0$ and ${\int }_0^{\mathrm{\infty }}{C}_A(s)ds={\sum }_{j\ge 1}{a}_j^2\mathrm{/}{\lambda }_j\le \mathrm{\parallel }{A}_c{\mathrm{\parallel }}_{L^2(\pi )}^2\mathrm{/}{\lambda }_{\ast }$. The process $A_c(X_s)$ is centered, jointly measurable and second-order stationary; the spectral sum makes its covariance absolutely integrable. By [F3] the variance of its time integral is the double-integral covariance formula. Dividing by $T^2$ and bounding $T-s\le T$ proves the stated variance bound.

For $t>0$, differentiate the finite sum defining $\mathcal{D}$; the mass derivative ${\sum }_a{p}_a^{\mathrm{\prime }}(t)$ is zero, so ${\mathcal{D}}^{\mathrm{\prime }}(t)={\sum }_a{p}_a^{\mathrm{\prime }}(t)\log r_a(t)={\sum }_a{\sum }_{b\mathrm{\ne }a}{p}_a(t)w_{ab}(\log r_b(t)-\log r_a(t))$. Use $p_a={\pi }_a{r}_a$ and pair the directed $(a,b)$ and $(b,a)$ terms with the symmetric conductance ${\pi }_a{w}_{ab}={\pi }_b{w}_{ba}$. This gives the displayed negative half-sum. For positive $x,y$, $(x-y)(\log x-\log y)\ge 0$, with equality only at $x=y$, since $\log x-\log y={\int }_y^{x}du\mathrm{/}u$. Connectedness forces all $r_a$ equal when the sum vanishes; normalization then gives $r_a=1$ and $p_t=\pi$.

Fix a reference time $t_{\ast }>0$ and put $T_n=n^2{t}_{\ast }$. For every $\epsilon >0$, the pointwise inequality ${\mathbf{1}}_{\{\mathrm{\mid }Y\mathrm{\mid }>\epsilon \}}\le Y^2\mathrm{/}{\epsilon }^2$ applied to $Y=T_n^{-1}{\int }_0^{T_n}{A}_c(X_s)ds$, together with step 1.2, gives ${\sum }_{n=1}^{\mathrm{\infty }}{\mathrm{Pr}}_{\pi }(\mathrm{\mid }Y\mathrm{\mid }>\epsilon )\le \frac{2\mathrm{\parallel }{A}_c{\mathrm{\parallel }}_{L^2(\pi )}^2}{{\lambda }_{\ast }{\epsilon }^2{t}_{\ast }}{\sum }_{n=1}^{\mathrm{\infty }}{n}^{-2}<\mathrm{\infty }$. For each $N$, the probability that one of these events occurs for some $n\ge N$ is at most the corresponding tail sum, which tends to zero. Thus only finitely many occur almost surely. Applying this argument to $\epsilon =1\mathrm{/}k$ for each positive integer $k$ proves convergence to zero along $T_n$, almost surely.

Let $M={\max }_{a\in E}\mathrm{\mid }{A}_c(a)\mathrm{\mid }$. For $T_n\le T\le T_{n+1}$, $\mathrm{\mid }{T}^{-1}{\int }_0^T{A}_c(X_s)ds-T_n^{-1}{\int }_0^{T_n}{A}_c(X_s)ds\mathrm{\mid }\le 2M(T_{n+1}-T_n)\mathrm{/}{T}_n\to 0$. Hence the time averages converge to zero for stationary initial law. For any future-path event $H$ and state $a$, let ${\mu }_a(H)$ be its probability under the common transition kernels started from $a$. Then ${\mathrm{Pr}}_{p_0}(H)={\sum }_a{p}_0(a){\mu }_a(H)={\mathbb{E}}_{\pi }[(p_0(X_0)\mathrm{/}\pi (X_0))1_H]$, so the forward path law from $p_0$ has density $p_0(X_0)\mathrm{/}\pi (X_0)$ relative to the stationary path law. This density is bounded because $E$ is finite and every ${\pi }_a>0$. Every stationary null event, including failure of the almost-sure limit, is therefore null for every initial preparation. Adding $\pi (A)$ proves the claimed limit.

Exponential convergence in [F2] gives $p_t\to \pi$ entrywise. The function $x\mapsto x\log (x\mathrm{/}{\pi }_a)$ extends continuously to $x=0$ by value zero, so the finite sum $\mathcal{D}(p_t\mathrm{\parallel }\pi )$ tends to zero. Combined with step 2.1 this proves all the relative-information claims.