Finite Pinsker inequality in Euclidean norm

If $P\ll ̸Q$, the first inequality is immediate because its left side is $+\mathrm{\infty }$. Otherwise let $A=\{i:p_i>q_i\}$ and $d=P(A)-Q(A)$. Since ${\sum }_i(p_i-q_i)=0$, the positive and negative coordinate differences have equal total magnitude, so $\mathrm{\parallel }P-Q{\mathrm{\parallel }}_1=2d$. If $A$ is empty, then $P=Q$ and the claim holds.

For $A$ and its complement, [F1] gives $D(P\mathrm{\parallel }Q)\ge p\ln (p\mathrm{/}q)+(1-p)\ln ((1-p)\mathrm{/}(1-q))$, where $p=P(A)$ and $q=Q(A)$, with endpoint terms interpreted by the same support convention.

If $q\in \{0,1\}$, finite divergence and $P\ll Q$ force $p=q$, so the binary divergence is zero and $d=0$. For $0<q<1$, define $g(p)=p\ln (p\mathrm{/}q)+(1-p)\ln ((1-p)\mathrm{/}(1-q))-2(p-q{)}^2$. On $0<p<1$, [F3] and the product rule give $g(q)=g^{\mathrm{\prime }}(q)=0$ and $g^{\mathrm{\prime }\mathrm{\prime }}(p)=1\mathrm{/}[p(1-p)]-4\ge 0$. Thus $g$ is convex and minimized at $q$; continuity of $t\ln t$ at zero from [F4] extends $g(p)\ge 0$ to $p=0,1$.

Therefore $D(P\mathrm{\parallel }Q)\ge 2d^2=\frac{1}{2}\mathrm{\parallel }P-Q{\mathrm{\parallel }}_1^2$. Since $\mathrm{\parallel }v{\mathrm{\parallel }}_2\le \mathrm{\parallel }v{\mathrm{\parallel }}_1$ for every finite vector $v$, the Euclidean bound follows.