Mutual information is recovered by finite coordinate quantizations

Let $({\mathcal{G}}_n)$ be any increasing sequence of sub-sigma-algebras generating $\mathcal{G}$, and let $\rho =P+Q$ for probability measures on $\mathcal{G}$. The union $\mathcal{U}={\bigcup }_n{\mathcal{G}}_n$ is an algebra. The class of $A\in \mathcal{G}$ approximable in $\rho$-measure by a set in $\mathcal{U}$ is a sigma-algebra: complements preserve the symmetric-difference measure, finite unions preserve approximation, and for a countable union one first truncates it using continuity from below [F5] of the finite measure $\rho$, then approximates the finite union. Hence every set in $\mathcal{G}$ is approximable. Approximating bounded measurable functions first by simple functions and then their level sets gives bounded $\mathcal{U}$-measurable functions converging in $L^1(\rho )$, with one ${\mathcal{G}}_n$ containing each finite simple approximation. The approximants may be clipped to the same bound M. On that bound, [F2]'s variational functional is continuous under $L^1(\rho )$ approximation: $e^t$ is Lipschitz on $[-M,M]$, and the exponential integral is bounded below by $e^{-M}$. By [F2] applied to $P,Q$ and their restrictions to each ${\mathcal{G}}_n$, the supremum over all bounded ${\mathcal{G}}_n$-measurable functions is $D(P{\mathrm{\mid }}_{{\mathcal{G}}_n}\mathrm{\parallel }Q{\mathrm{\mid }}_{{\mathcal{G}}_n})$. These test classes increase, so the restricted divergences increase; the approximation shows their supremum equals the unrestricted variational supremum $D(P\mathrm{\parallel }Q)$. Hence $D(P\mathrm{\parallel }Q)={\lim }_{n}D(P{\mathrm{\mid }}_{{\mathcal{G}}_n}\mathrm{\parallel }Q{\mathrm{\mid }}_{{\mathcal{G}}_n})$.

By [F1], countable generating algebras exist; their first-n finite Boolean subalgebras yield increasing finite quantizers by labeling their atoms. For the quantizers in the Statement, the same generated-sigma property is assumed, so in either case $\sigma (q_n(X),r_n(Y))$ increases to the product sigma-algebra of $(X,Y)$. Under the map $(x,y)\mapsto (q_n(x),r_n(y))$, the push-forward of $P_X\otimes P_Y$ is $P_{q_n(X)}\otimes P_{r_n(Y)}$, as is checked on rectangles of the finite output sets. Since $\sigma (q_n(X),r_n(Y))$ consists exactly of preimages of output events, every bounded measurable test function on that sigma-algebra factors through the finite output map. Therefore its restricted KL equals the KL of the push-forward laws, which by [F3] is $I(q_n(X);r_n(Y))$. Applying step 1.1 proves the ordinary mutual-information limit.

For the conditional statement, put ${\mathcal{G}}_n=\sigma (Z,q_n(X),r_n(Y))$. These sigma-algebras increase to the full product sigma-algebra of $(Z,X,Y)$ because the entire Z sigma-algebra is retained. A bounded ${\mathcal{G}}_n$-measurable simple function factors through $(Z,q_n(X),r_n(Y))$: each level set is the preimage of a measurable output set, and finitely many output sets can be made disjoint without changing their preimages. Uniform simple approximation extends this factorization to all bounded variational test functions. Hence the KL restricted to ${\mathcal{G}}_n$ equals the KL between the push-forward joint and reference laws. Under the same map, the push-forward of the reference law in [F4] has, given Z=z, the product of the push-forward kernels $K_z\circ q_n^{-1}$ and $L_z\circ r_n^{-1}$: on a rectangle $C\times A^{\mathrm{\prime }}\times B^{\mathrm{\prime }}$ its mass is ${\int }_C{K}_z(q_n^{-1}{A}^{\mathrm{\prime }})L_z(r_n^{-1}{B}^{\mathrm{\prime }})P_Z(dz)$. By rectangle uniqueness, this is exactly the conditional-product reference law for the quantized variables. Step 1.1 therefore gives the conditional limit.

Steps 2.1 and 2.2 establish both limits. The proof quantizes X and Y only; it makes no claim that coarsening Z decreases conditional mutual information.