LemmaStatement: Literature-sourcedProof: AI-adaptedPipeline-generatedprecheck pass

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Continuity of finite conditional entropy in the joint law

Statement

For fixed finite nonempty alphabets $U, V$ , the map from a joint probability vector $q_{U V}$ on $U \times V$ to its conditional entropy $H_{q} (U ∣ V)$ is continuous on the joint probability simplex.

Facts & Assumptions

Given: The finite alphabets $U, V$ and a joint probability vector $q_{U V}$ .

[F1]

Finite conditional entropy: $H_{q} (U ∣ V) = \sum_{v : q_{V} (v) > 0} q_{V} (v) H (q_{U ∣ v})$ , with zero-mass conditioning values omitted.

[F2]

Marginals and positive-mass conditionals of a finite joint law are probability vectors: the marginals of a finite joint law are probability vectors, and each positive-mass conditional vector is a probability vector.

[F3]

Finite distributions, entropy, divergence, and stochastic matrices: $H (r) = - \sum_{i : r_{i} > 0} r_{i} \ln r_{i}$ and $D (r ∥ s) = \sum_{i : r_{i} > 0} r_{i} \ln (r_{i} / s_{i})$ , with $0 \ln 0 = 0$ .

[F4]

Finite relative entropy is continuous on a simplex with positive reference law: if $s$ is a fixed full-support probability vector on a finite alphabet, then $r \mapsto D (r ∥ s)$ is finite and continuous on the full simplex, including its boundary.

Proof

1.1F3F4algebra

Let $r$ be a probability vector on an alphabet of size $m \geq 1$ , and let $u_{m}$ be the uniform vector. Expanding the finite sum in [F3] gives $D (r ∥ u_{m}) = \ln m - H (r)$ , including zero coordinates by the stated convention. Since $u_{m}$ has full support, [F4] shows that $r \mapsto D (r ∥ u_{m})$ is continuous. Therefore finite entropy is continuous on every finite simplex.

2.1F1F2F3step 1.1algebra∎

For any joint vector $q_{U V}$ , [F1] and [F2] give $H_{q} (U ∣ V) = - \sum_{(u, v) : q_{U V} (u, v) > 0} q_{U V} (u, v) \ln \frac{q_{U V} (u, v)}{q_{V} (v)} = H (q_{U V}) - H (q_{V})$ . The ratio is defined on every positive joint atom because then $q_{V} (v) > 0$ . The marginal map $q_{U V} \mapsto q_{V}$ is linear. By step 1.1, both terms on the right are continuous functions of $q_{U V}$ , so their difference is continuous on the joint simplex.

Depends on

Used by

Conditional typical-set fan cardinality bound Lemma

Dependency tree · two levels

10 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Yury Polyanskiy and Yihong Wu, Information Theory: From Coding to Learning, author-hosted book draft (standard reference, not scraped)
Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, second edition (standard reference, not scraped)