GAN objectives and the ideal discriminator

Define $\psi (0)=1$ and $\psi (t)=t\ln t-t+1$ for $t>0$. If $0<t<1$, then $\ln t=-{\int }_t^{1}du\mathrm{/}u>-(1-t)\mathrm{/}t=1-1\mathrm{/}t$, since $1\mathrm{/}u<1\mathrm{/}t$ for $t<u\le 1$. If $t>1$, then $\ln t={\int }_1^{t}du\mathrm{/}u>(t-1)\mathrm{/}t=1-1\mathrm{/}t$, since $1\mathrm{/}u>1\mathrm{/}t$ for $1\le u<t$. Thus $\psi (t)>0$ for $t\mathrm{\ne }1$ and $\psi (1)=0$. Also $-1\le \varphi (t)\le 0$ for $0<t\le 1$, by the first inequality and $\ln t\le 0$; and $0\le \varphi (t)\le t(t-1)\le 2$ for $1\le t\le 2$, since $\ln t={\int }_1^{t}du\mathrm{/}u\le t-1$. At $t=0$, $\varphi (0)=0$ and $\psi (0)=1$. Hence $\psi \ge 0$ with equality exactly at $1$, $\mathrm{\mid }\varphi (t)\mathrm{\mid }\le 2$, and $\psi (t)\le 5$ on $[0,2]$.

By [F1], [F7] and [F8], $\int md\mu =\frac{1}{2}\int (p+q)d\mu =1$. By [F2], $0\le r,s\le 2$. The ratio maps $R(a,b)=2a\mathrm{/}(a+b)$ and $S(a,b)=2b\mathrm{/}(a+b)$ on $a+b>0$, extended by $0$ at $(0,0)$, are Borel because they are continuous on $\{a+b>0\}$ and separately defined on the Borel singleton $\{(0,0)\}$. The functions $\varphi ,\psi$ are continuous on $[0,2]$, including at $0$. Thus coordinatewise measurability [F10], composition with these Borel maps [F11], and arithmetic closure [F9] give measurable $m,r,s,\varphi (r),\varphi (s),\psi (r),\psi (s)$; product closure [F12] then gives measurability of $m\varphi (r)$, $m\varphi (s)$, and $\frac{1}{2}m(\psi (r)+\psi (s))$. Step 1.1 gives $\mathrm{\mid }m\varphi (r)\mathrm{\mid },\mathrm{\mid }m\varphi (s)\mathrm{\mid }\le 2m$ and $0\le \frac{1}{2}m(\psi (r)+\psi (s))\le 5m$, so they are integrable. On $m>0$, $r+s=2$ and $\varphi (t)=\psi (t)+t-1$, so $\varphi (r)+\varphi (s)=\psi (r)+\psi (s)\ge 0$; when $m=0$ both products by $m$ vanish. Linearity in [F6] gives ${\mathrm{JS}}_{\mu }(P,Q)=\int \frac{1}{2}m(\psi (r)+\psi (s))d\mu \ge 0$.

If ${\mathrm{JS}}_{\mu }(P,Q)=0$, the nonnegative measurable function $h:=\frac{1}{2}m(\psi (r)+\psi (s))$ has integral zero by step 2.1. By [F5], $h=0$ $\mu$-almost everywhere. On $m>0$, both $\psi (r)$ and $\psi (s)$ are nonnegative, so they must each be zero almost everywhere there; step 1.1 gives $r=s=1$ there, hence $p=q$. On $m=0$, nonnegativity and $m=(p+q)\mathrm{/}2$ give $p=q=0$. Thus $p=q$ $\mu$-almost everywhere. For each $E\in \mathcal{A}$, [F1] and [F13] give $P(E)-Q(E)=\int p{\chi }_{E}d\mu -\int q{\chi }_{E}d\mu$. These restrictions are integrable because they are bounded by $p$ and $q$, whose integrals are $1$; [F8] and [F6] rewrite their difference as $\int u_{E}d\mu$ for $u_E:=p{\chi }_E-q{\chi }_E$. The positive and negative parts of $u_E$ are measurable by [F9, F12] and vanish almost everywhere, so [F5] makes both nonnegative integrals zero. By [F14], $\int u_{E}d\mu =0$, hence $P(E)=Q(E)$.

Conversely, suppose $P=Q$. For positive integers $n,k$, put $E_{n,k}:=\{p-q\ge 1\mathrm{/}n\}\cap \{p+q\ge 1\mathrm{/}k\}$, which is measurable by [F9]. Since $\{p>q\}={\bigcup }_{n,k\ge 1}{E}_{n,k}$, positive measure of $\{p>q\}$ would make some $E:=E_{n,k}$ have positive measure. On $E$, $(1\mathrm{/}k){\chi }_E\le (p+q){\chi }_E\le p+q$, so [F8], [F13], [F15] and [F16] give $\mu (E)\mathrm{/}k\le {\int }_E(p+q)d\mu \le \int (p+q)d\mu =2$ by [F1], [F7] and step 2.1; hence $\mu (E)<\mathrm{\infty }$. Put $u:=(p-q){\chi }_E$. By [F9, F12], $u$ is nonnegative measurable, and $(1\mathrm{/}n){\chi }_E\le u$, so [F8], [F15] and [F16] give $\int ud\mu \ge \mu (E)\mathrm{/}n>0$. Also $p{\chi }_E$ and $q{\chi }_E$ are integrable since they are bounded by $p$ and $q$; [F1], [F13] and [F6] therefore give $\int ud\mu =P(E)-Q(E)=0$, a contradiction. The same argument with $p,q$ exchanged rules out $\{q>p\}$, hence $p=q$ $\mu$-almost everywhere. Then $m\varphi (r)$ and $m\varphi (s)$ vanish almost everywhere: on $m>0$, $r=s=1$ almost everywhere, and on $m=0$ both products are zero. Their positive and negative parts are measurable by [F9, F12] and vanish almost everywhere, so [F5] makes all four nonnegative integrals zero. Step 2.1 gives their integrability, and [F14] and [F2] therefore yield ${\mathrm{JS}}_{\mu }(P,Q)=0$.

Steps 3.1 and 3.2 prove that ${\mathrm{JS}}_{\mu }(P,Q)$ is zero exactly when $P=Q$; step 2.1 proves it is finite and nonnegative. These are all claims in the Statement.

Define $F:[0,\mathrm{\infty }{)}^2\to [0,1]$ by $F(a,b)=a\mathrm{/}(a+b)$ when $a+b>0$ and $F(0,0)=1\mathrm{/}2$. It is continuous on the Borel set $\{a+b>0\}$ and constant on its singleton complement, hence Borel. By [F8], $x\mapsto (p(x),q(x))$ is measurable, and [F9] makes $D^{\ast }=F(p,q)$ measurable.

Fix $x$ and write $a=p(x)$ and $b=q(x)$. If $a,b>0$, the function $f(y)=a\ln y+b\ln (1-y)$ on $0<y<1$ is differentiable with $f^{\mathrm{\prime }}(y)=a\mathrm{/}y-b\mathrm{/}(1-y)=(a-(a+b)y)\mathrm{/}(y(1-y))$: the derivative of $1-y$ is $-1$ by its difference quotient, [F4] differentiates $\ln$, [F10] applies the chain rule, and [F11] gives the sum and scalar rules. The denominator is positive; the numerator is positive for $y<y^{\ast }:=a\mathrm{/}(a+b)$ and negative for $y>y^{\ast }$. By [F12], $f$ is continuous on every closed subinterval of $(0,1)$. For any $y\in (0,y^{\ast })$, [F13] on $[y,y^{\ast }]$ gives $c\in (y,y^{\ast })$ with $f(y^{\ast })-f(y)=f^{\mathrm{\prime }}(c)(y^{\ast }-y)>0$. For any $y\in (y^{\ast },1)$, [F13] on $[y^{\ast },y]$ gives $c\in (y^{\ast },y)$ with $f(y)-f(y^{\ast })=f^{\mathrm{\prime }}(c)(y-y^{\ast })<0$. At $y=0,1$ the payoff is $-\mathrm{\infty }$. If $a=0<b$, then $f(y)=b\ln (1-y)\le 0=f(0)$, with equality at $y=0$. If $b=0<a$, then $f(y)=a\ln y\le 0=f(1)$, with equality at $y=1$. If $a=b=0$, every $y$ gives value $0$. In all four cases, $D^{\ast }(x)$ maximizes the pointwise payoff.

For any measurable $D$, step 1.2 gives the pointwise loss inequality $p(-\ln D)+q(-\ln (1-D))\ge p(-\ln D^{\ast })+q(-\ln (1-D^{\ast }))$, with zero-density endpoint terms interpreted as in [F1]. These are nonnegative measurable losses by the definition and its measurability dependencies [F1]. Additivity [F14] and monotonicity [F6] therefore give $\int p(-\ln D)d\mu +\int q(-\ln (1-D))d\mu \ge \int p(-\ln D^{\ast })d\mu +\int q(-\ln (1-D^{\ast }))d\mu$. Taking negatives and using [F1] yields $V(D;P,Q)\le V(D^{\ast };P,Q)$. Since $D^{\ast }$ is measurable by step 1.1, the supremum is attained and $C(P,Q)=V(D^{\ast };P,Q)$.

On $m>0$, one has $p=mr$, $q=ms$, $D^{\ast }=r\mathrm{/}2$ and $1-D^{\ast }=s\mathrm{/}2$. Using $0\ln 0=0$ when $r=0$ or $s=0$, the pointwise optimized payoff is $m\varphi (r)+m\varphi (s)-m(r+s)\ln 2=m\varphi (r)+m\varphi (s)-2m\ln 2$. On $m=0$, both densities and all displayed products are zero, so the same identity holds. The first two terms are integrable by [F3]; [F1] and [F7] give $\int md\mu =1$. Hence [F7], [F2] and $\ln 4=2\ln 2$ [F5] give $C(P,Q)=2{\mathrm{JS}}_{\mu }(P,Q)-\ln 4$.

By [F3], ${\mathrm{JS}}_{\mu }(P,Q)\ge 0$, and it vanishes exactly when $P=Q$. Step 3.1 therefore gives $C(P,Q)\ge -\ln 4$ with equality exactly at $P=Q$. The unrestricted density class contains the data law itself, so this value is attained there and no other generator law attains it. The argument establishes only the distribution-space optimum stated here.