Finite data processing for Rényi divergence

Suppose $\alpha >1$. If $P\ll ̸Q$, [F1] gives $D_{\alpha }(P\mathrm{\parallel }Q)=+\mathrm{\infty }$, so the claimed inequality holds. If $P\ll Q$, then $PK\ll QK$: whenever $(QK{)}_y=0$, nonnegativity of the summands gives $q_{x}K(y\mid x)=0$ for every $x$, and $p_x>0$ implies $q_x>0$ and hence $K(y\mid x)=0$. For $f(t)=t^{\alpha }-1$, convexity holds, $f_{\mathrm{\infty }}=+\mathrm{\infty }$ by [F5], and direct substitution into [F2] gives $D_f(P\mathrm{\parallel }Q)=S_{\alpha }(P,Q)-1$ and the same identity for $PK,QK$.

Suppose $0<\alpha <1$ and take $g(t)=1-t^{\alpha }$. By [F5] it is convex, $g(1)=0$, $g(0)=1$, and $g_{\mathrm{\infty }}=0$. Substitution into [F2] gives $D_g(P\mathrm{\parallel }Q)=1-S_{\alpha }(P,Q)$, including when $P$ has mass outside $\mathrm{supp}Q$; likewise $D_g(PK\mathrm{\parallel }QK)=1-S_{\alpha }(PK,QK)$.

By [F4], $S_{\alpha }(PK,QK)-1\le S_{\alpha }(P,Q)-1$. The logarithm is increasing and $\alpha -1>0$, so dividing the logarithms by $\alpha -1$ preserves the inequality and proves the result for $\alpha >1$.

By [F4], $S_{\alpha }(PK,QK)\ge S_{\alpha }(P,Q)$. If the input affinity is zero, [F1] gives $D_{\alpha }(P\mathrm{\parallel }Q)=+\mathrm{\infty }$ and the conclusion follows. Otherwise both affinities are positive; applying the increasing logarithm and then dividing by $\alpha -1<0$ reverses the inequality. This proves the claim for $0<\alpha <1$.