Average-cost fixed-length codes and operational DMC capacity

Fix a finite DMC and cost model from Finite DMC with a nonnegative input cost and feasible budget, with feasible budget $\mathrm{\Gamma }$. A length-$n$, $M$-message code from Fixed-length codes for a finite DMC is average-cost feasible if $\frac{1}{nM}{\sum }_{m=1}^M{\sum }_{t=1}^{n}c(x_t(m))\le \mathrm{\Gamma }\mathrm{.}$ A rate $R\ge 0$ is average-error achievable under budget $\mathrm{\Gamma }$ if there is a sequence of such codes, for all sufficiently large $n$, whose average errors from Average and maximal error of a fixed-length code tend to zero and satisfy ${\mathrm{lim\hspace{0.17em}inf}}_{n\to \mathrm{\infty }}{n}^{-1}\ln M_n\ge R$. The operational cost-constrained capacity $C(\mathrm{\Gamma })$ is the supremum of these rates.

For an input probability vector $P_X$, define its expected cost $\bar{c}(P_X)$ as in Finite DMC with a nonnegative input cost and feasible budget. The cost-constrained information capacity is $C_I(\mathrm{\Gamma }):={\sup }_{P_X:\bar{c}(P_X)\le \mathrm{\Gamma }}I(P_X;W),$ where $I(P_X;W)$ and the unconstrained DMC information capacity use natural-log nats as in Single-letter information capacity of a finite DMC and The natural logarithm as the inverse of the exponential function. All rates use natural-log nats (The natural logarithm as the inverse of the exponential function). This definition uses a supremum; attainment is proved separately.