Time reversal of a stationary Markov chain

If $x\in E_{+}$ and $y\mathrm{\notin }{E}_{+}$ then $p(x,y)=0$: otherwise $\pi (y)={\sum }_z\pi (z)p(z,y)\ge \pi (x)p(x,y)>0$ by [F1], contradicting $\pi (y)=0$. Hence ${\sum }_{y\in E_{+}}p(x,y)=1$ for every $x\in E_{+}$.

The formula $p_{\ast }(x,y)=\pi (y)p(y,x)\mathrm{/}\pi (x)$ is well defined for $x\in E_{+}$ because $\pi (x)>0$, and nonnegative; for $x\in E_{+}$ its row sum is ${\sum }_{y\in E}{p}_{\ast }(x,y)=\frac{1}{\pi (x)}{\sum }_y\pi (y)p(y,x)=\frac{(\pi p)(x)}{\pi (x)}=\frac{\pi (x)}{\pi (x)}=1$, using [F1]; since $\pi (y)=0$ for $y\mathrm{\notin }{E}_{+}$, the extended entries $p_{\ast }(x,y)$ vanish off $E_{+}$, so $p_{\ast }$ is a transition matrix on $E_{+}$ and no mass leaves $E_{+}$.

The probability $\pi$ restricted to $E_{+}$ is invariant for $p_{\ast }$: for $y\in E_{+}$, ${\sum }_{x\in E_{+}}\pi (x)p_{\ast }(x,y)={\sum }_{x\in E_{+}}\pi (y)p(y,x)=\pi (y){\sum }_{x\in E_{+}}p(y,x)=\pi (y)$ by step 1.1, while for $y\mathrm{\notin }{E}_{+}$ both sides vanish; this is precisely invariance in the countable form [F1].

Detailed balance equivalence: for $x,y\in E_{+}$ the identity $\pi (x)p(x,y)=\pi (y)p(y,x)$ is equivalent, after dividing by the positive number $\pi (x)$, to $p(x,y)=\pi (y)p(y,x)\mathrm{/}\pi (x)=p_{\ast }(x,y)$; if $x\in E_{+}$ and $y\mathrm{\notin }{E}_{+}$, then $\pi (x)p(x,y)=0$ by step 1.1 and $\pi (y)p(y,x)=0$ because $\pi (y)=0$. The case $x\mathrm{\notin }{E}_{+}$, $y\in E_{+}$ follows by exchanging $x$ and $y$; if both states are outside $E_{+}$, both weights vanish. Transitions from outside $E_{+}$ into $E_{+}$ need not vanish. Hence detailed balance for $\pi$ and $p$ holds for all pairs exactly when $p_{\ast }=p$ on $E_{+}\times E_{+}$.

By [F5] construct a $p_{\ast }$-chain $X^{\ast }$ on $E_{+}$ with initial law $\pi {\mathrm{\mid }}_{E_{+}}$; it is stationary by [F2] and step 2.1. Consecutive-time reversal: for $n\ge 0$ and states $x_0,\dots ,x_n\in E_{+}$, [F3] with $\mu =\pi$ (and indicators) gives ${\mathbb{P}}_{\pi }(X_0=x_0,\dots ,X_n=x_n)=\pi (x_0){\prod }_{i=0}^{n-1}p(x_i,x_{i+1})$; reading the same word backward and using the definition of $p_{\ast }$, $\pi (x_n){\prod }_{i=0}^{n-1}{p}_{\ast }(x_{i+1},x_i)=\pi (x_n){\prod }_{i=0}^{n-1}\frac{\pi (x_i)p(x_i,x_{i+1})}{\pi (x_{i+1})}=\pi (x_0){\prod }_{i=0}^{n-1}p(x_i,x_{i+1})$ after telescoping cancellation of the $\pi (x_i)$, $i=1,\dots ,n-1$; paths visiting $E\setminus E_{+}$ have probability zero by step 1.1, so $\mathcal{L}(X_n,\dots ,X_0)=\mathcal{L}(X_0^{\ast },\dots ,X_n^{\ast })$ for a stationary $p_{\ast }$-chain $X^{\ast }$ with initial law $\pi$, by [F3] and step 2.1.

For every $r\ge 1$ and $0\le n_0<\cdots <n_r$, step 3.1 at $n=n_r$ gives $\mathcal{L}(X_{n_r},\dots ,X_0)=\mathcal{L}(X_0^{\ast },\dots ,X_{n_r}^{\ast })$. Taking the coordinates indexed by $0,n_r-n_{r-1},\dots ,n_r-n_0$ on both sides yields the asserted law of $(X_{n_r},\dots ,X_{n_0})$.

Null rows: since $\pi (x)=0$ for $x\mathrm{\notin }{E}_{+}$, such a state carries no stationary mass and does not appear in the reversal statements of items 1–3, which only involve paths with positive probability; if a kernel on all of $E$ is wanted, fix $y_0\in E_{+}$ (the set $E_{+}$ is nonempty because $\pi$ is a probability) and set $p_{\ast }(x,\cdot ):={\delta }_{y_0}$ for $x\mathrm{\notin }{E}_{+}$, which is a probability row and leaves every assertion about $E_{+}$ unchanged.

Boundary and axiom cases: if $E_{+}=E$ (the chain is irreducible and positive recurrent, or more generally $\pi$ has full support) then no null rows arise and clause 3 compares the two kernels on all of $E\times E$; if $E$ is a singleton, $p_{\ast }=p=1$ and both reversal and detailed balance are trivial; the reversal identity of step 3.1 is symmetric in the two directions and does not presuppose $p_{\ast }=p$, so the statement covers nonreversible stationary chains; AC [A1] is used in constructing $X^{\ast }$ through [F5], proving its stationarity through [F2], and computing finite-dimensional laws through [F3]; and all products and telescoping cancellations are finite, no infinite sum being rearranged.