KL chain rule for standard-Borel disintegrations

If $\mu \ll ̸\nu$, choose A with $\nu (A)=0<\mu (A)$. Then $Q(A\times F)=0<P(A\times F)$ by [F1], so $P\ll ̸Q$. Both $D(P\mathrm{\parallel }Q)$ and $D(\mu \mathrm{\parallel }\nu )+D_{\mu }(K\mathrm{\parallel }L)$ are $+\mathrm{\infty }$. Now suppose $\mu \ll \nu$ but $P\ll ̸Q$. If $D_{\mu }(K\mathrm{\parallel }L)<\mathrm{\infty }$, [F5] implies $D(K(x,\cdot )\mathrm{\parallel }L(x,\cdot ))<\mathrm{\infty }$ for $\mu$-almost every x, hence $K(x,\cdot )\ll L(x,\cdot )$ there by [F6]. For any product-measurable C with $Q(C)=0$, [F1] and [F5] give $L(x,C_x)=0$ for $\nu$-almost every x, hence for $\mu$-almost every x. On the intersection with the conditional absolute-continuity set, $K(x,C_x)=0$, and then [F1] gives $P(C)=0$, a contradiction. Thus $D_{\mu }(K\mathrm{\parallel }L)=+\mathrm{\infty }$ and the identity holds in this case too.

It remains to consider $P\ll Q$. Then $\mu \ll \nu$, since $Q(A\times F)=0$ whenever $\nu (A)=0$, and [F1] gives $P(A\times F)=\mu (A)$. Choose finite-valued nonnegative measurable representatives $a=d\mu \mathrm{/}d\nu$ and $h=dP\mathrm{/}dQ$ by [F3]. In particular, $\mu (\{a=0\})={\int }_{\{a=0\}}ad\nu =0.$ For each B in the countable determining algebra ${\mathcal{A}}_F$ of [F4] and every measurable A in E, [F1] and [F3] give ${\int }_{A}a(x)K(x,B)\nu (dx)=P(A\times B)={\int }_A({\int }_{B}h(x,y)L(x,dy))\nu (dx)\mathrm{.}$ The two integrands are measurable and nonnegative, and their integrals over E are finite. Their equality on every A therefore makes them equal $\nu$-almost everywhere. Intersect the corresponding measurable conull sets over the countable algebra to obtain T. Since $\mu \ll \nu$, the set $S=T\cap \{a>0\}$ has full $\mu$-measure. Let $\rho (t)=1\mathrm{/}t$ for $t>0$ and $\rho (0)=0$; this is Borel on $[0,+\mathrm{\infty })$. The first projection ${\pi }_E:E\times F\to E$ is measurable because ${\pi }_E^{-1}(A)=A\times F$. Define $r(x,y)={\mathbf{1}}_{S\times F}(x,y)h(x,y)\rho (a({\pi }_E(x,y)))\mathrm{.}$ By [F8], $r$ is finite-valued, nonnegative and product-measurable; it equals $h(x,y)\mathrm{/}a(x)$ for $x\in S$ and zero elsewhere. For each $x\in S$ the preceding identities give $K(x,B)={\int }_{B}r(x,y)L(x,dy)(B\in {\mathcal{A}}_F)\mathrm{.}$ The algebra contains F, so the right-hand measure has total mass one. Measure determination [F4] extends the equality to every measurable B: $r(x,\cdot )$ is a Radon–Nikodym representative of $K(x,\cdot )$ with respect to $L(x,\cdot )$ for every $x\in S$. No conditional identity outside the $\mu$-full set S is needed.

For product-measurable C, let $g_C(x)={\int }_F{\mathbf{1}}_C(x,y)r(x,y)L(x,dy)$, which is measurable by [F1]. The fiber identity from step 1.2 and the joint-law formula for Q give $P(C)=\int K(x,C_x)\mu (dx)=\int g_C(x)\mu (dx)=\int a(x)g_C(x)\nu (dx)={\int }_{C}a(x)r(x,y)Q(d(x,y)),$ where [F7] justifies the change from $\mu$ to its density $a\nu$. Thus the globally measurable nonnegative function $ar$ is a Radon–Nikodym density of P with respect to Q; uniqueness in [F3] gives $h=ar$ Q-almost everywhere, hence P-almost everywhere because $P\ll Q$. Moreover, $P(\{h=0\})={\int }_{\{h=0\}}hdQ=0$, and $P(\{a=0\}\times F)=\mu (\{a=0\})=0$. Intersecting these conull sets with $S\times F$ shows that $a,h,r$ are all strictly positive and finite on one P-full set. Therefore $\ln h=\ln a+\ln r$ there; on the exceptional zero sets use the zero log-density convention of [F6]. By [F6], the negative part of $\ln a$ under $\mu$ is at most $1\mathrm{/}e$, and for $\mu$-almost every x the negative part of $\ln r(x,\cdot )$ under $K(x,\cdot )$ is at most $1\mathrm{/}e$; integrating the latter bound over the probability base $\mu$ is finite. Hence both sides can be integrated by adding their positive and negative parts separately, with finite negative parts and no $\mathrm{\infty }-\mathrm{\infty }$ ambiguity. The section formula [F1] now gives $D(P\mathrm{\parallel }Q)=\int \ln adP+\int \ln rdP=D(\mu \mathrm{\parallel }\nu )+\int D(K(x,\cdot )\mathrm{\parallel }L(x,\cdot ))\mu (dx)\mathrm{.}$ The right side is a sum of extended nonnegative divergences, so this equality includes the infinite cases.