Variational and finite-partition formulas for measurable-space KL

First suppose $P\ll Q$ and $D(P\mathrm{\parallel }Q)<\mathrm{\infty }$. For bounded measurable $f$, let $Z_f=\int e^{f}dQ$ and define $Q_f$ by $d{Q}_f=e^f{Z}_f^{-1}dQ$. Then $Q_f$ is a probability equivalent to $Q$, and $dP\mathrm{/}d{Q}_f=(dP\mathrm{/}dQ)Z_f{e}^{-f}$. Taking logarithms and integrating under $P$ gives $D(P\mathrm{\parallel }{Q}_f)=D(P\mathrm{\parallel }Q)-{\mathrm{\Phi }}_{P,Q}(f)$. By [F2], ${\mathrm{\Phi }}_{P,Q}(f)\le D(P\mathrm{\parallel }Q)$. If $D(P\mathrm{\parallel }Q)=+\mathrm{\infty }$ or $P\ll ̸Q$, this upper bound is automatic.

If $P\ll ̸Q$, choose a measurable $A$ with $Q(A)=0<P(A)$. For $f_t=t{\mathbf{1}}_A$, $\int e^{f_t}dQ=1$ and ${\mathrm{\Phi }}_{P,Q}(f_t)=tP(A)\to +\mathrm{\infty }$. Thus the bounded-function supremum is $+\mathrm{\infty }=D(P\mathrm{\parallel }Q)$ in the singular case.

Suppose $P\ll Q$, let $r=dP\mathrm{/}dQ$, and set $\mathrm{\ell }=\log r$ on $\{r>0\}$, with $\mathrm{\ell }=-\mathrm{\infty }$ on $\{r=0\}$; the latter set is P-null. Define $f_n=\max (-n,\min (n,\mathrm{\ell }))$, taking $f_n=-n$ on $\{r=0\}$. The negative part of $\mathrm{\ell }$ is integrable by [F1]; its clipped negative parts converge by [F4], while the clipped positive parts increase to ${\mathrm{\ell }}^{+}$ and converge in integral by [F4]. Hence $\int f_{n}dP\to D(P\mathrm{\parallel }Q)$, including value $+\mathrm{\infty }$. Pointwise, $e^{f_n}\to r$ Q-a.e. and $0\le e^{f_n}\le r+1$, so [F4] gives $\int e^{f_n}dQ\to \int rdQ=1$. Therefore ${\mathrm{\Phi }}_{P,Q}(f_n)\to D(P\mathrm{\parallel }Q)$ and the bounded-function supremum is at least the divergence.

Fix a finite measurable partition $(A_i{)}_{i=1}^m$ and put $p_i=P(A_i)$, $q_i=Q(A_i)$. For constants $c_i$, the simple function $f={\sum }_i{c}_i{\mathbf{1}}_{A_i}$ has $\mathrm{\Phi }(f)={\sum }_i{p}_i{c}_i-\ln {\sum }_i{q}_i{e}^{c_i}$. If some $p_i>0=q_i$, this expression is unbounded above by taking $c_i\to +\mathrm{\infty }$. Otherwise its supremum over the $c_i$ is ${\sum }_{i:p_i>0}{p}_i\ln (p_i\mathrm{/}{q}_i)$: the upper bound follows by applying [F2] to the finite probability vector with masses proportional to $q_i{e}^{c_i}$, and the lower bound follows by taking $c_i=\ln (p_i\mathrm{/}{q}_i)$ when $p_i>0$ and $c_i\to -\mathrm{\infty }$ on cells with $p_i=0<q_i$. The cases $p_i=q_i=0$ do not affect either sum.

Every bounded measurable $f$ is uniformly approximated by finite-valued measurable simple functions. If $\mathrm{\parallel }f-g{\mathrm{\parallel }}_{\mathrm{\infty }}\le \epsilon$, then $\mathrm{\mid }\int fdP-\int gdP\mathrm{\mid }\le \epsilon$ and $e^{-\epsilon }\int e^{g}dQ\le \int e^{f}dQ\le e^{\epsilon }\int e^{g}dQ$, so $\mathrm{\mid }\mathrm{\Phi }(f)-\mathrm{\Phi }(g)\mathrm{\mid }\le 2\epsilon$. Thus the supremum over bounded functions equals the supremum over simple functions. Step 2.2 identifies the latter with the supremum over finite partitions; step 2.1 identifies it with $D(P\mathrm{\parallel }Q)$. No step subtracts infinite quantities.