Fano's inequality for a finite message observed through a density

Since $\int qd\nu =N^{-1}{\sum }_m\int p_{m}d\nu =1$, [F6] makes $B=\{q=+\mathrm{\infty }\}$ a measurable $\nu$-null set. Replace every $p_m$ by zero on $B$ and use their new mixture $q$; [F7] shows that all conditional and marginal laws are unchanged. These density versions are finite everywhere, since $p_m\le Nq$ off $B$. Define ${\pi }_m=p_m\mathrm{/}(Nq)$ where $q>0$ and ${\pi }_m=1\mathrm{/}N$ where $q=0$. This is a measurable probability vector everywhere and a posterior version: for every measurable $A$, ${\int }_{A}q{\pi }_{m}d\nu =N^{-1}{\int }_A{p}_{m}d\nu =\mathrm{Pr}(M=m,Y\in A)$. Values on the zero-density set have no effect. Thus $H(M\mid Y)=\int H(\pi )dQ$, where $dQ=qd\nu$ is the observation probability law. By [F1] and comparison with the uniform vector in [F2], $0\le H(\pi )\le \ln N$, so this conditional entropy is finite. All posterior formulas in the Statement are understood for these almost-everywhere density versions.

For each decoder value $j$ with ${\alpha }_j=Q(A_j)>0$, let $A_j=g^{-1}(j)$ and $r^{(j)}={\alpha }_j^{-1}{\int }_{A_j}\pi dQ$; this is the posterior probability vector given that estimate. If $r_m^{(j)}=0$, then ${\int }_{A_j}{\pi }_{m}dQ=0$, so [F5], applied to the measure $Q$ restricted to $A_j$, gives ${\pi }_m=0$ almost everywhere there. Discard the union of these finitely many exceptional null sets. On the remainder, $\pi$ is supported on the positive coordinates of $r^{(j)}$, so $D(\pi \mathrm{\parallel }{r}^{(j)})$ has no infinite term. The finitely many constants $\ln r_m^{(j)}$ on that support and the bounded entropy in step 1.1 justify termwise integration. Expansion and [F2] give ${\alpha }_j^{-1}{\int }_{A_j}D(\pi \mathrm{\parallel }{r}^{(j)})dQ=H(r^{(j)})-{\alpha }_j^{-1}{\int }_{A_j}H(\pi )dQ\ge 0$. Decoder cells of zero $Q$-mass contribute zero; summing over the other cells proves $H(M\mid g(Y))\ge H(M\mid Y)$.

Apply [F3] to the finite pair $(M,g(Y))$. Its sharp and relaxed Fano bounds give both upper inequalities in the Statement; the entropy convention in [F4] makes them nats. Together with step 2.1 this proves the full chain.