Finite entropy and mutual-information chain rules

Fix $z$ with $p_Z(z)>0$ and write $q_{xy}=p_{XY\mathrm{\mid }z}(x,y)$, $q_x={\sum }_y{q}_{xy}$; for $q_x>0$ define $q_{y\mathrm{\mid }x}=q_{xy}\mathrm{/}{q}_x$. By [F4], these are pmfs and $q_{xy}=q_x{q}_{y\mathrm{\mid }x}$ on positive support. On that support both factors are positive, so the product law [F6] gives $-{\sum }_{q_{xy}>0}{q}_{xy}\ln q_{xy}=-{\sum }_{q_x>0}{\sum }_{y:q_{y\mathrm{\mid }x}>0}{q}_x{q}_{y\mathrm{\mid }x}(\ln q_x+\ln q_{y\mathrm{\mid }x})$. For every $q_x>0$, ${\sum }_{y:q_{y\mathrm{\mid }x}>0}{q}_{y\mathrm{\mid }x}=1$. Thus the terms involving $\ln q_x$ sum to $-{\sum }_{q_x>0}{q}_x\ln q_x$, and the remaining terms are ${\sum }_{q_x>0}{q}_x(-{\sum }_{q_{y\mathrm{\mid }x}>0}{q}_{y\mathrm{\mid }x}\ln q_{y\mathrm{\mid }x})$, exactly $H(q_X)+{\sum }_x{q}_{x}H(q_{Y\mathrm{\mid }x})$ by [F5]. Weighting this equality by $p_Z(z)$ and summing over positive-mass $z$ using [F1] proves $H(X,Y\mathrm{\mid }Z)=H(X\mathrm{\mid }Z)+H(Y\mathrm{\mid }X,Z)$. Zero atoms are omitted from the displayed logarithmic sums.

Every positive joint atom has positive marginals by [F4]; applying the quotient and product laws [F6] to [F2] gives $I(U;V)={\sum }_{p_{uv}>0}{p}_{uv}(\ln p_{uv}-\ln p_U(u)-\ln p_V(v))$. Grouping the finite sums by $u,v$, using [F4] to know the marginals, and applying [F5] yields $I(U;V)=H(U)+H(V)-H(U,V)$. Also, [F1] and $p_{U\mathrm{\mid }v}=p_{UV}\mathrm{/}{p}_V$ on positive support give $H(U\mathrm{\mid }V)=-{\sum }_{p_{uv}>0}{p}_{uv}\ln (p_{uv}\mathrm{/}{p}_V(v))=H(U,V)-H(V)$. Thus $I(U;V)=H(U)-H(U\mathrm{\mid }V)$, and exchanging $U,V$ gives the other form. All sums are finite.

Apply step 1.1 and [F3] to get $I(X;Y\mathrm{\mid }Z)=H(Y\mathrm{\mid }Z)-H(Y\mathrm{\mid }X,Z)$. By step 1.2, $H(Y,Z)=H(Z)+H(Y\mathrm{\mid }Z)$ and $H(X,Y,Z)=H(Z)+H(X,Y\mathrm{\mid }Z)=H(X,Z)+H(Y\mathrm{\mid }X,Z)$. Therefore $I(X;Y,Z)=H(X)+H(Y,Z)-H(X,Y,Z)=I(X;Z)+I(X;Y\mathrm{\mid }Z)$. Every entropy is a finite sum, so no extended-real subtraction occurs.