Finite relative-entropy chain rule and event grouping

Write $p_x=P_X(x)$ and $q_x=Q_X(x)$. Strict positivity of $Q$ gives $q_x>0$ and $Q_{Y\mathrm{\mid }x}(y)>0$ for each $x,y$. If $P(x,y)>0$, then $p_x>0$ and $\ln \frac{P(x,y)}{Q(x,y)}=\ln \frac{p_x}{q_x}+\ln \frac{P_{Y\mathrm{\mid }x}(y)}{Q_{Y\mathrm{\mid }x}(y)}$ by [F2]. Multiply by $P(x,y)$ and sum over positive joint atoms. For each $x$ with $p_x>0$, summing the first term over $y$ gives $p_x\ln (p_x\mathrm{/}{q}_x)$, and summing the second gives $p_{x}D(P_{Y\mathrm{\mid }x}\mathrm{\parallel }{Q}_{Y\mathrm{\mid }x})$. Atoms with $p_x=0$ contribute no terms. This is exactly the claimed chain rule.

If $C$ is empty or all of $I\times J$, then $P(C)=Q(C)$ is respectively $0$ or $1$, so the binary divergence is $0$ under the stated zero-weight convention. The grouping bound is then $D(P\mathrm{\parallel }Q)\ge 0$, with equality exactly when $P=Q$ by [F1]. For a nonempty proper $C$, strict positivity gives $0<Q(C)<1$. Put $a=P(C)$ and $b=Q(C)$. When $a>0$, the conditional laws on $C$ are $P_C(z)=P(z)\mathrm{/}a$ and $Q_C(z)=Q(z)\mathrm{/}b$; the latter is strictly positive on $C$. The logarithmic identity $\ln (P(z)\mathrm{/}Q(z))=\ln (a\mathrm{/}b)+\ln (P_C(z)\mathrm{/}{Q}_C(z))$ for each $z\in C$ with $P(z)>0$ yields ${\sum }_{z\in C:P(z)>0}P(z)\ln \frac{P(z)}{Q(z)}=a\ln \frac{a}{b}+aD(P_C\mathrm{\parallel }{Q}_C)\mathrm{.}$ If $a=0$, both the sum and the weighted conditional term are zero, so the same identity holds with that term omitted. Applying the identical calculation to $C^c$ and adding gives $D(P\mathrm{\parallel }Q)=d(a\mathrm{\parallel }b)+aD(P_C\mathrm{\parallel }{Q}_C)+(1-a)D(P_{C^c}\mathrm{\parallel }{Q}_{C^c})$, with zero-weight conditional terms omitted. Both weighted conditional divergences are nonnegative by [F1], which proves the grouping inequality, including $a=0$ or $1$. Equality holds precisely when every conditional divergence with positive weight vanishes; by the equality clause of [F1], this is equivalent to agreement of the conditional probability vectors on each positive-$P$-mass block. Zero-$P$-mass blocks impose no conditional-law condition.