Gallager and expurgated input-law maxima are attained

For any fixed $a>0$, the convention in [F3] extends $t\mapsto t^a$ continuously to $t=0$: given $ϵ>0$, put $\delta =\exp ((\ln ϵ)\mathrm{/}a)>0$. If $0<t<\delta$, monotonicity of $\ln$ gives $a\ln t<\ln ϵ$, so monotonicity of $\exp$ yields $0<t^a<ϵ$; at $t=0$ the value is zero. Thus every positive real power used in [F1] is continuous on $[0,\mathrm{\infty })$.

For every $x$, $B(x,x)={\sum }_{y}W(y\mid x)=1$ by [F2]. Thus the inner sum in $E_{0x}$ satisfies ${\sum }_{x,x^{\mathrm{\prime }}}P(x)P(x^{\mathrm{\prime }})B(x,x^{\mathrm{\prime }}{)}^{1\mathrm{/}\rho }\ge {\sum }_{x}P(x{)}^2\ge \frac{1}{\mathrm{\mid }\mathcal{X}\mathrm{\mid }}>0.$ The last inequality follows from $\mathrm{\mid }\mathcal{X}\mathrm{\mid }{\sum }_{x}P(x{)}^2-({\sum }_{x}P(x){)}^2={\sum }_{x<x^{\mathrm{\prime }}}(P(x)-P(x^{\mathrm{\prime }}){)}^2\ge 0$. Write $d=\mathrm{\mid }\mathcal{X}\mathrm{\mid }$, $c_{x{x}^{\mathrm{\prime }}}=B(x,x^{\mathrm{\prime }}{)}^{1\mathrm{/}\rho }$, $C={\sum }_{x,x^{\mathrm{\prime }}}{c}_{x{x}^{\mathrm{\prime }}}>0$, and $T(P)={\sum }_{x,x^{\mathrm{\prime }}}{c}_{x{x}^{\mathrm{\prime }}}P(x)P(x^{\mathrm{\prime }})$. For input laws $P,Q$, their coordinates lie in $[0,1]$, so $\mathrm{\mid }T(P)-T(Q)\mathrm{\mid }\le {\sum }_{x,x^{\mathrm{\prime }}}{c}_{x{x}^{\mathrm{\prime }}}(\mathrm{\mid }P(x)-Q(x)\mathrm{\mid }+\mathrm{\mid }P(x^{\mathrm{\prime }})-Q(x^{\mathrm{\prime }})\mathrm{\mid })\le 2C\mathrm{\parallel }P-Q{\mathrm{\parallel }}_2\mathrm{.}$ Thus $T$ is continuous on the simplex. At each $P$, scalar continuity of $\ln$ at $T(P)\ge 1\mathrm{/}d$ supplies a tolerance $\eta >0$ such that $\mathrm{\mid }t-T(P)\mathrm{\mid }<\eta$ and $t>0$ imply $\rho \mathrm{\mid }\ln t-\ln T(P)\mathrm{\mid }<ϵ$. Taking $\mathrm{\parallel }P-Q{\mathrm{\parallel }}_2<\eta \mathrm{/}(2C)$ proves continuity of $E_{0x}(\rho ,P)=-\rho \ln T(P)$ directly, without applying [F4] on a vector domain.

Fix $\rho >0$ and set $A_y(P)={\sum }_{x}P(x)W(y\mid x{)}^{1\mathrm{/}(1+\rho )}$ and $S(P)={\sum }_y{A}_y(P{)}^{1+\rho }$. Since the fixed coefficients in $A_y$ lie in $[0,1]$, $\mathrm{\mid }{A}_y(P)-A_y(Q)\mathrm{\mid }\le d\mathrm{\parallel }P-Q{\mathrm{\parallel }}_2$. At a fixed $P$ and for $ϵ>0$, step 1.1 supplies, for each of the finitely many $y$, a scalar tolerance ${\eta }_y>0$ such that $\mathrm{\mid }t-A_y(P)\mathrm{\mid }<{\eta }_y$ with $t\ge 0$ implies $\mathrm{\mid }{t}^{1+\rho }-A_y(P{)}^{1+\rho }\mathrm{\mid }<ϵ\mathrm{/}\mathrm{\mid }\mathcal{Y}\mathrm{\mid }$. Taking $\mathrm{\parallel }P-Q{\mathrm{\parallel }}_2<{\min }_y{\eta }_y\mathrm{/}d$ and summing proves $\mathrm{\mid }S(P)-S(Q)\mathrm{\mid }<ϵ$. For every input law $P$, some $x$ has $P(x)>0$; since row $W(\cdot \mid x)$ sums to one, some $y$ has $W(y\mid x)>0$, giving $A_y(P)>0$ and hence $S(P)>0$. Scalar logarithm continuity at $S(P)$, followed by the just-proved continuity estimate for $S$, therefore gives continuity of the real-valued $E_0(\rho ,P)=-\ln S(P)$. At $\rho =0$, the sum is ${\sum }_y{\sum }_{x}P(x)W(y\mid x)=1$, so $E_0(0,P)=0$.

The finite probability vectors form the nonempty simplex $\mathrm{\Delta }=\{P:P(x)\ge 0,{\sum }_{x}P(x)=1\}$ by [F5]. It is nonempty because a point mass belongs to it; each coordinate lies in $[0,1]$, so it is bounded; and the coordinate inequalities are closed half-spaces while the sum equation is a closed affine hyperplane, so $\mathrm{\Delta }$ is closed. Heine–Borel makes $\mathrm{\Delta }$ compact, and steps 1.2 and 2.1 show that each objective is continuous and real-valued on it. The extreme-value assertion in [F5] therefore gives an attained finite maximum for every stated $\rho$.