Finite-alphabet conditional entropy is version-invariant and obeys its chain rule

If two versions of the conditional law of $X$ given $Z$ are supplied, [F2] gives one $P_Z$-null set outside which their probability measures agree. Every atom probability, and hence the finite entropy sum in [F1], then agrees pointwise outside that set. Integrating gives the same $H(X\mid Z)$. The same argument applies to $(X,U)\mid Z$ and to $U\mid (X,Z)$.

Choose regular conditional laws of $(X,U)$ given $Z$ and of $U$ given $(X,Z)$; they exist by [A1, F3], since $(X,U)$ and $(X,Z)$ are standard Borel by [F4]. Write $p_{ij}(z)=P(X=i,U=j\mid Z=z)$, $p_i(z)={\sum }_j{p}_{ij}(z)$, and $m_{ij}(z)=P(U=j\mid X=i,Z=z)$ for the chosen versions. The function $p_i$ is itself a conditional law of $X$ given $Z$, so [F2] makes it agree almost everywhere with the version used in $H(X\mid Z)$. For each $i,j$ and measurable $C$ in the value space of $Z$, the two disintegrations [F3] give ${\int }_C{p}_i(z)m_{ij}(z)P_Z(dz)=P(X=i,U=j,Z\in C)={\int }_C{p}_{ij}(z)P_Z(dz)\mathrm{.}$ Both integrands are bounded and measurable. Testing the sets where one exceeds the other by at least $1\mathrm{/}n$, for each $n\ge 1$, shows $p_i(z)m_{ij}(z)=p_{ij}(z)$ almost everywhere; because the alphabets are finite, one null set works for every $i,j$.

Outside that null set, if $p_i(z)=0$ then every $p_{ij}(z)=0$, so the row contributes zero to the conditional entropy. If $p_i(z)>0$, then $m_{ij}(z)=p_{ij}(z)\mathrm{/}{p}_i(z)$ for every $j$. On positive entries, $\ln p_{ij}=\ln p_i+\ln m_{ij}$; multiplying by $-p_{ij}$ and summing over the finite alphabets gives $H(X,U\mid Z=z)=H(X\mid Z=z)+{\sum }_i{p}_i(z)H(U\mid X=i,Z=z)\mathrm{.}$

Integrate the finite conditional identity from step 2.1 with respect to $P_Z$. The disintegration identity [F3] identifies the last integral with $H(U\mid X,Z)$ and the other terms with their definitions. By the bound $-t\ln t\le 1\mathrm{/}e$ in [F1], each conditional entropy is bounded by its finite alphabet size divided by $e$, so all integrals are finite and no subtraction of infinite quantities occurs. Step 1.1 proves the accompanying version-independence assertion.