Ionescu-Tulcea construction of a Markov chain

Construct prefix probabilities recursively. Given ${\mu }_n$ on [F1] $E_0\times \cdots \times E_n$, define ${\mu }_{n+1}(A)=\int K_n(x_0,\dots ,x_n,A_{(x_0,\dots ,x_n)})d{\mu }_n\mathrm{.}$ The integrand is measurable by [F1]. For disjoint $A_j$, sectionwise countable additivity and [F8] move the increasing partial sums through the outer integral; the empty set has mass $0$ and the whole product has mass $1$. Thus ${\mu }_{n+1}$ is a probability measure. Since $K_n(x,E_{n+1})=1$, its marginal on the first $n+1$ coordinates is ${\mu }_n$. Induction gives consistent prefix laws and hence consistent laws for arbitrary finite coordinate sets by marginalization.

The spaces are nonempty along a compatible history: ${\mu }_0(E_0)=1$, and a [F2, step 1.1] probability section of each $K_n$ has nonempty target. Choice supplies one compatible infinite coordinate sequence. Together with the prefix construction in step 1.1, this shows the product is nonempty and [F2] defines a finitely additive probability $P_0$ on the cylinder algebra $\mathcal{A}$.

Starting from the cylinder law in step 2.1, we prove continuity at the empty set, the missing premeasure condition. Let [F1, F5, step 2.1] $C_j\downarrow \mathrm{\varnothing }$ be cylinders and suppose instead that $P_0(C_j)\downarrow a>0$. Represent $C_j$ using the first $r_j+1$ coordinates, enlarging so that $r_j$ increases. For a prefix $x^{(k)}=(x_0,\dots ,x_k)$ and $j$ with $r_j\ge k$, let $p_j^{(k)}(x^{(k)})$ be the probability, under the remaining kernels through time $r_j$, that the completed prefix lies in $C_j$. These functions are measurable by repeated [F1], lie in $[0,1]$, and decrease in $j$. Put $m_k={\lim }_j{p}_j^{(k)}$. Dominated convergence [F5] gives the recursion $m_k(x^{(k)})=\int m_{k+1}(x^{(k)},y)K_k(x^{(k)},dy),$ while another use of [F5] gives $a=\int m_0(x_0){\mu }_0(d{x}_0)$.

Since $a>0$, some $x_0$ has $m_0(x_0)>0$. Whenever [step 3.1] $m_k(x^{(k)})>0$, the recursion in step 3.1 implies that some $x_{k+1}$ has $m_{k+1}(x^{(k)},x_{k+1})>0$. Choice selects these coordinates recursively. For fixed $j$, once the selected prefix reaches $r_j$, monotonicity gives $0<m_{r_j}(x^{(r_j)})\le p_j^{(r_j)}(x^{(r_j)})=1_{C_j}(x^{(r_j)})\mathrm{.}$ Thus the selected infinite point belongs to every $C_j$, contradicting their empty intersection. Therefore $P_0(C_j)\downarrow 0$. This is the exact nonempty-choice use in the proof; no compactness or tail measure is assumed.

If disjoint cylinders $A_j$ have cylinder union $A$, then [F3, F4, step 4.1] $R_N=A\setminus {\bigcup }_{j<N}{A}_j$ is a decreasing cylinder sequence with empty intersection. Finite additivity and step 4.1 give $P_0(A)={\sum }_{j<N}{P}_0(A_j)+P_0(R_N)⟶{\sum }_j{P}_0(A_j)\mathrm{.}$ Hence, by [F3], $P_0$ is a finite premeasure. Since Choice implies countable choice, [F4] extends it to a probability $P$ on the product sigma-algebra.

If $P^{\mathrm{\prime }}$ is another extension, it agrees with $P$ on every cylinder by the [F6, step 5.1] prescribed finite laws. Cylinders form a pi-system containing the whole product and generate the product sigma-algebra, so [F6] gives $P^{\mathrm{\prime }}=P$. This also handles zero cylinder events and the total-mass-one cylinder.

Under the probability $P$ constructed and identified in step 6.1, in the homogeneous last-coordinate specialization, let [F7, step 1.1, step 6.1] ${\mathcal{F}}_n=\sigma (X_0,\dots ,X_n)$. The recursive prefix law in step 1.1 says for every $B\in {\mathcal{F}}_n$ and $A\in \mathcal{E}$ that ${\mathbb{E}}_P[1_B{1}_{\{X_{n+1}\in A\}}]={\mathbb{E}}_P[1_{B}K(X_n,A)]\mathrm{.}$ The right-hand random variable is ${\mathcal{F}}_n$-measurable, so it is the conditional probability. Hence [F7] says that the coordinates form the homogeneous Markov chain and they have initial law ${\mu }_0$.