Finite-action average total variation is bounded by average KL

Fix one state $s$. Write ${\mathrm{\Delta }}_a=q_s(a)-p_s(a)$; since both distributions have mass one, ${\sum }_a{\mathrm{\Delta }}_a=0$. The set $B=\{a:{\mathrm{\Delta }}_a\ge 0\}$ consists of the nonnegative differences. The positive and negative parts of $\mathrm{\Delta }$ therefore have equal sum, each equal to $\frac{1}{2}{\sum }_a\mathrm{\mid }{\mathrm{\Delta }}_a\mathrm{\mid }$. For every subset $C\subseteq A(s)$, $\mathrm{\mid }{\sum }_{a\in C}{\mathrm{\Delta }}_a\mathrm{\mid }$ is at most this common sum, and $C=B$ attains it. Thus the event-supremum total variation in [F2] equals ${\mathrm{TV}}_{A(s)}(q_s,p_s)$.

Apply [F3] to the probability measures $q_s$ and $p_s$ on the finite power-set space, and use step 1.1. This gives ${\mathrm{TV}}_{A(s)}(q_s,p_s)\le \sqrt{D_{\mathrm{K}\mathrm{L}}(q_s\mathrm{\parallel }{p}_s)\mathrm{/}2}$, including the infinite-divergence case.

If $K=+\mathrm{\infty }$, the asserted inequality holds because each total-variation term is at most one. Suppose $K<\mathrm{\infty }$ and put $m=K\mathrm{/}2$. When $m=0$, step 2.1 gives $D_{\mathrm{K}\mathrm{L}}(q_s\mathrm{\parallel }{p}_s)\ge 2{\mathrm{TV}}_{A(s)}(q_s,p_s{)}^2\ge 0$ for every state. Since their $d$-weighted sum is zero, every state with $d(s)>0$ has zero divergence, so step 2.1 also gives zero total variation there. When $m>0$, for every $x\ge 0$ the square $(\sqrt{x}-\sqrt{m}{)}^2\ge 0$ gives $\sqrt{x}\le (\sqrt{m}+x\mathrm{/}\sqrt{m})\mathrm{/}2$. Apply this with $x=D_{\mathrm{K}\mathrm{L}}(q_s\mathrm{\parallel }{p}_s)\mathrm{/}2$ for states with $d(s)>0$, multiply by $d(s)$, and sum. Since ${\sum }_{s}d(s)=1$ and ${\sum }_{s}d(s)x_s=m$, the result is ${\sum }_{s}d(s)\sqrt{x_s}\le \sqrt{m}$. Step 2.1 now yields the claimed bound.