TheoremStatement: Literature-sourcedProof: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Finite-alphabet Fano inequality

Statement

Let $X$ take values in an alphabet $X$ of size $m \geq 2$ , let $Y$ be finite, and let $\hat{X}$ be an estimator in the same alphabet generated from $Y$ , so the joint pmf factors as $p_{X Y \hat{X}} = p_{X Y} p_{\hat{X} ∣ Y}$ . Put $P_{e} = P [\hat{X} \neq X]$ and define

$h_{2} (t) : = - t \ln t - (1 - t) \ln (1 - t), 0 \ln 0 : = 0.$

Then

$H (X ∣ Y) \leq h_{2} (P_{e}) + P_{e} \ln (m - 1) \leq \ln 2 + P_{e} \ln m .$

For $m = 1$ , $X$ is constant and $H (X ∣ Y) = 0$ .

Facts & Assumptions

Given: A finite joint pmf of $X, Y, \hat{X}$ with $p_{X Y \hat{X}} = p_{X Y} p_{\hat{X} ∣ Y}$ and $∣ X ∣ = m \geq 1$ .

[F1]

Data processing for a finite Markov chain, with equality condition: if $X \to Y \to \hat{X}$ , then $I (X; \hat{X}) \leq I (X; Y)$ .

[F2]

Finite entropy and mutual-information chain rules: $I (X; Y) = H (X) - H (X ∣ Y)$ and finite conditional entropy obeys the chain rule.

[F3]

Finite KL divergence is nonnegative with equality exactly at equal laws: finite KL divergence is nonnegative, with equality iff its two probability vectors agree.

[F4]

Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm: $\ln$ is increasing on $(0, \infty)$ and $\ln 1 = 0$ .

[F5]

Finite distributions, entropy, divergence, and stochastic matrices: $H (r) = - \sum_{r_{i} > 0} r_{i} \ln r_{i}$ , with $0 \ln 0 = 0$ .

Proof

1.1givenF1F2F5algebracases

If $m = 1$ , then $X$ is constant and $H (X ∣ Y) = 0$ . Hence assume $m \geq 2$ . The factorization in the hypotheses is the Markov condition $X \to Y \to \hat{X}$ . By [F1], $I (X; \hat{X}) \leq I (X; Y)$ ; subtracting from the finite value $H (X)$ and using [F2] gives $H (X ∣ Y) \leq H (X ∣ \hat{X})$ .

1.2F2F5givenalgebra

Set $E = 1 {X \neq \hat{X}}$ . Since $E$ is determined by $(X, \hat{X})$ , [F2] gives $H (X ∣ \hat{X}) = H (E ∣ \hat{X}) + H (X ∣ E, \hat{X})$ . The law of $E$ is Bernoulli with mass $P_{e}$ at $1$ , so [F5] gives $H (E) = h_{2} (P_{e})$ .

2.1F2F3step 1.2algebra

The finite MI identity in [F2] gives $H (E) - H (E ∣ \hat{X}) = I (E; \hat{X})$ . By [F3], this MI, which is KL between the joint law and product marginals, is nonnegative. Thus $H (E ∣ \hat{X}) \leq h_{2} (P_{e})$ .

2.2F3step 1.2givenalgebra

On $E = 0$ , $X = \hat{X}$ , so the conditional entropy of $X$ given $(E, \hat{X})$ is zero. On $E = 1$ and a fixed value of $\hat{X}$ , at most $m - 1$ values of $X$ remain. For any pmf $r$ on at most $k = m - 1$ points, pad by zeros to a $k$ -vector and compare with the uniform vector $u_{k}$ : [F3] gives $0 \leq D (r ∥ u_{k}) = \ln k - H (r)$ , so $H (r) \leq \ln (m - 1)$ . Averaging these conditional entropies yields $H (X ∣ E, \hat{X}) \leq P_{e} \ln (m - 1)$ .

3.1step 1.1step 1.2step 2.1step 2.2algebra

Combining steps 1.2, 2.1, and 2.2 gives $H (X ∣ \hat{X}) \leq h_{2} (P_{e}) + P_{e} \ln (m - 1)$ ; step 1.1 then gives the stated sharp bound.

4.1F3F4F5givenstep 3.1algebra∎

Since $D ((P_{e}, 1 - P_{e}) ∥ (1 / 2, 1 / 2)) = \ln 2 - h_{2} (P_{e}) \geq 0$ by [F3], $h_{2} (P_{e}) \leq \ln 2$ . Also $m - 1 \leq m$ and [F4] makes $\ln (m - 1) \leq \ln m$ , so step 3.1 implies the weaker bound. If $P_{e} = 0$ or $1$ , zero-probability conditional branches above are omitted and $h_{2} (P_{e}) = 0$ by convention.

Depends on

Used by

Dependency tree · two levels

17 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Robert M. Gray, Entropy and Information Theory, corrected first edition (standard reference, not scraped)
Yury Polyanskiy and Yihong Wu, Information Theory: From Coding to Learning, author prepublication (standard reference, not scraped)