Gibbs sampling and scan schemes — Examples

The coordinate marginals are uniform on $\{0,1\}$. For coordinate 1 set $K_{\{1\}}(t,\cdot )={\delta }_t$; for any measurable $C\subseteq S$ and $D\subseteq \{0,1\}$, $\pi (C\cap p_2^{-1}(D))=\frac{1}{2}{\sum }_{t\in D}{\mathbf{1}}_C(t,t)={\int }_D{K}_{\{1\}}(t,C_t){\pi }_2(dt)$, where $C_t=\{s:(s,t)\in C\}$. Exchanging the coordinates gives the same identity for $K_{\{2\}}(t,\cdot )={\delta }_t$. Each section is a probability measure at every conditioning value, and the identities for all $C,D$ prove that these are proper full conditional kernels by [F3].

Starting from either state $(t,t)\in E$, updating coordinate 1 draws $t$ from $K_{\{1\}}(t,\cdot )$, and updating coordinate 2 draws $t$ from $K_{\{2\}}(t,\cdot )$. Thus both component transition matrices restricted to $E$ are the identity. Each singleton $\{(0,0)\}$ and $\{(1,1)\}$ is nonempty, proper, and closed, so [F2] makes the restricted finite chain reducible.

Apply [F1] with $m_X=m_Y=0$, ${\sigma }_X={\sigma }_Y=1$, and $r=\rho$. This gives $X\mid Y=y\sim N(\rho y,1-{\rho }^2)$ for every $y$. Applying the same supplier with the coordinates exchanged gives $Y\mid X=x\sim N(\rho x,1-{\rho }^2)$ for every $x$. Since $1-{\rho }^2>0$, both laws are nondegenerate normal probabilities, and the supplier gives their normalized densities as everywhere conditional kernels.

With these two kernels, [F2] updates $X$ by drawing from the first conditional while holding $Y$ fixed, and then updates $Y$ from the second conditional at the new $X$ value. Both are exact block updates for the stated target, with no support or zero-normalizer exception because the conditional variances are strictly positive.

The conditional means in [F1], integrated in the order [F6] under the joint law [F5], give $\mathbb{E}{X}_1=\rho \mathbb{E}{Y}_0=0$ and $\mathbb{E}{Y}_1=\rho \mathbb{E}{X}_1=0$. By [F8], the initial target has $\mathbb{E}{Y}_0^2=1$ and $\mathbb{E}[X_0{Y}_0]=\rho$. The first conditional variance in [F1] gives $\mathbb{E}[X_1^2\mid Y_0]={\rho }^2{Y}_0^2+1-{\rho }^2$, so $\mathbb{E}{X}_1^2=1$. The invariant full-sweep law [F2] gives $(X_1,Y_1)\sim \pi$, hence $\mathbb{E}{Y}_1^2=1$. Thus all four coordinates are square-integrable, and their cross-products are integrable by $2\mathrm{\mid }ab\mathrm{\mid }\le a^2+b^2$.

The means in step 1.1 are zero, so [F4] makes each cross-covariance equal to its product expectation. Using the kernel order [F6], the first conditional mean in [F1], and [F8], $\mathrm{Cov}(Y_0,X_1)=\int \pi (d{x}_0,d{y}_0)y_0\int x_1{K}_1(y_0,d{x}_1)=\rho \mathbb{E}{Y}_0^2=\rho$. For the other cross term, the second conditional mean followed by the first gives $\mathrm{Cov}(X_0,Y_1)=\int \pi (d{x}_0,d{y}_0)x_0\int K_1(y_0,d{x}_1)\int y_1{K}_2(x_1,d{y}_1)={\rho }^2\mathbb{E}[X_0{Y}_0]={\rho }^3$. The integrability established in step 1.1 justifies these signed iterated integrals.

If the full-sweep kernel were reversible, [F3] would give equality of its two-time law and the state-swapped measure on every rectangle. Both are finite measures, so [F7] gives equality on the product sigma-algebra. The integrable function $((x_0,y_0),(x_1,y_1))\mapsto y_0{x}_1$ would then have the same integral as its state-swapped version $x_0{y}_1$, forcing $\mathrm{Cov}(Y_0,X_1)=\mathrm{Cov}(X_0,Y_1)$ because all four coordinates are centered. But [F8] and step 2.1 give these covariances as $\rho$ and ${\rho }^3$, which differ for $\rho =\frac{1}{2}$. Hence the full-sweep kernel is not reversible, although [F2] proves it preserves $\pi$.