Kullback–Leibler divergence is nonnegative

If $P\ll ̸Q$, [F1] gives $D(P\mathrm{\parallel }Q)=+\mathrm{\infty }$. If $P\ll Q$ but $D(P\mathrm{\parallel }Q)=+\mathrm{\infty }$, nonnegativity is immediate. It remains to consider $P\ll Q$ with finite divergence; let $r=dP\mathrm{/}dQ$.

Put $\phi (0)=0$ and $\phi (t)=t\log t$ for $t>0$. By [F5] and [F7], ${\phi }^{\mathrm{\prime }\mathrm{\prime }}(t)=1\mathrm{/}t>0$ for $t>0$, so [F8] makes $\phi$ convex on $(0,\mathrm{\infty })$. Its continuity at zero is part of [F1]; taking limits in the convexity inequality as a positive endpoint tends to zero extends convexity to $[0,\mathrm{\infty })$.

Define $g(t):=t\log t-t+1$ for $t>0$ and $g(0):=1$. For $s>1$, set $c=(1+s)\mathrm{/}2$. By [F11], $\log s={\int }_1^{c}du\mathrm{/}u+{\int }_c^{s}du\mathrm{/}u$. On $[1,c]$, $1\mathrm{/}u\le 1$, while on $[c,s]$, $1\mathrm{/}u\le 1\mathrm{/}c<1$; [F9] and [F24] give $\log s<s-1$. For $0<s<1$, set $c=(1+s)\mathrm{/}2$. By [F11], $\log s=-{\int }_s^{1}du\mathrm{/}u=-({\int }_s^{c}du\mathrm{/}u+{\int }_c^{1}du\mathrm{/}u)$. On $[s,c]$, $1\mathrm{/}u\ge 1\mathrm{/}c>1$, and on $[c,1]$, $1\mathrm{/}u\ge 1$; [F9] and [F24] give $\log s<s-1$. The integrands and comparison constants are integrable on these compact intervals by [F10]. Equality holds at $s=1$. Applying this inequality to $s=1\mathrm{/}t$ and using [F6] gives $t\log t\ge t-1$, strictly unless $t=1$. Hence $g\ge 0$ and its only zero is $1$.

For the normalized-density clause on the measure space [F13], put $P:=q\mu$ and $Q:=\pi \mu$. By [F17], $\int \pi d\mu =Z^{-1}\int \gamma d\mu =1$; hence [F15] and [F14] make $P$ and $Q$ probability measures.

If $Q(E)=0$, [F15] gives ${\int }_E\pi d\mu =0$, hence ${\int }_E\gamma d\mu =0$ because $Z$ is finite and positive, using [F17]. By [F12], $\gamma =0$ $\mu$-almost everywhere on $E$. The support hypothesis on $q$ then gives $P(E)=0$, so $P\ll Q$. Define $\rho :[0,\mathrm{\infty }]\to [0,\mathrm{\infty }]$ by $\rho (t)=1\mathrm{/}t$ for finite $t>0$ and $\rho (0)=\rho (+\mathrm{\infty })=0$. For $a<0$, $\{\rho >a\}=[0,\mathrm{\infty }]$; for $a=0$ it is $(0,\mathrm{\infty })$; and for $a>0$ it is $(0,1\mathrm{/}a)$. These are Borel by [F18], so [F19] makes $\rho$ Borel. By [F20]–[F21], $r:=q\rho (\pi )$ is measurable and equals $q\mathrm{/}\pi$ on $\{\pi >0\}$ and zero on $\{\pi =0\}$. For every measurable $E$, [F16] gives ${\int }_{E}rdQ={\int }_{E}r\pi d\mu ={\int }_{E}qd\mu =P(E),$ because $q=0$ $\mu$-almost everywhere on $\{\pi =0\}$. Thus $r=dP\mathrm{/}dQ$.

By [F2] with $E$ equal to the whole space, $\int rdQ=P(\mathrm{\Omega })=1$. The negative part of $r\log r$ is finite by [F1], so finiteness of $D(P\mathrm{\parallel }Q)$ also makes its positive part finite. Thus $r$ and $r\log r$ are integrable with respect to the probability measure $Q$. The Radon–Nikodym representative here is the one whose selection is covered by [A1].

On the $P$-full-measure subset of $\{q>0\}$ where $\gamma >0$, we have $\pi =\gamma \mathrm{/}Z>0$ and $\mathrm{\ell }:=\log r=\log q-\log \gamma +\log Z$. Where $q=0$, the KL logarithm $\mathrm{\ell }$ is defined to be zero. By [F23], $q\mathrm{\mid }\mathrm{\ell }\mathrm{\mid }\le q\mathrm{\mid }\log q\mathrm{\mid }+q\mathrm{\mid }\log \gamma \mathrm{\mid }+q\mathrm{\mid }\log Z\mathrm{\mid }$ almost everywhere. The right-hand side has finite integral by the hypotheses and $\int q=1$; hence [F16] gives $\int \mathrm{\mid }\mathrm{\ell }\mathrm{\mid }dP=\int q\mathrm{\mid }\mathrm{\ell }\mathrm{\mid }d\mu <\mathrm{\infty }$. By [F1], [F16], and [F22], applying [F16] to the positive and negative parts of $\mathrm{\ell }$ gives $D(P\mathrm{\parallel }Q)=\int \log rdP=\int q(\log q-\log \gamma +\log Z)d\mu \mathrm{.}$ Using $\int qd\mu =1$ and rearranging proves the identity.

Apply [F3] on $(\mathrm{\Omega },\mathcal{A},Q)$ to $r$ and $\phi$. Steps 2.1 and 1.2 check its integrability and convexity hypotheses, and $\int rdQ=1$ by step 2.1. Therefore $0=\phi (1)=\phi \text{\hspace{0.17em}⁣}(\int rdQ)\le \int \phi (r)dQ=D(P\mathrm{\parallel }Q)\mathrm{.}$

The representative $r$ is finite-valued and measurable by [F1], and $\phi (r)$ is measurable there; the arithmetic measurability result [F21] therefore makes $g(r)=\phi (r)-r+1$ measurable. It is nonnegative by step 1.3. By step 2.1, $r,\phi (r)\in L^1(Q)$, and $1\in L^1(Q)$ because $Q$ is a probability measure. Linearity [F25] now gives $g(r)\in L^1(Q)$ and $\int g(r)dQ=\int \phi (r)dQ-\int rdQ+\int 1dQ=D(P\mathrm{\parallel }Q)-1+1=D(P\mathrm{\parallel }Q)\mathrm{.}$ Thus if $D(P\mathrm{\parallel }Q)=0$, [F12] applies to the measurable nonnegative integrand and gives $g(r)=0$ $Q$-almost everywhere. By step 1.3, $r=1$ $Q$-almost everywhere, and the Radon–Nikodym identity [F2] gives $P(E)=Q(E)$ for every measurable $E$. Conversely, if $P=Q$ then $r=1$ almost everywhere and [F1] gives $D(P\mathrm{\parallel }Q)=0$.

Step 2.2 identifies the normalized-density divergence as finite, and step 3.1 proves it is nonnegative. Therefore $\int q(\log \gamma -\log q)d\mu \le \log Z$, as claimed.