TheoremStatement: Literature-sourcedProof: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Finite KL divergence is nonnegative with equality exactly at equal laws

Statement

For finite probability vectors $p, q$ on the same alphabet, $D (p ∥ q) \geq 0$ , and $D (p ∥ q) = 0$ if and only if $p = q$ . If $p_{i} > 0 = q_{i}$ for some coordinate, then $D (p ∥ q) = + \infty$ .

Facts & Assumptions

Given: Finite probability vectors $p, q$ on a common alphabet.

[F1]

Finite distributions, entropy, divergence, and stochastic matrices: $D (p ∥ q) = \sum_{i : p_{i} > 0} p_{i} \ln (p_{i} / q_{i})$ , with $+ \infty$ if $p_{i} > 0 = q_{i}$ for some $i$ .

[F2]

Finite log-sum inequality with support and equality cases: for nonnegative vectors $a, b$ with positive sums $A, B$ , $\sum_{a_{i} > 0} a_{i} \ln (a_{i} / b_{i}) \geq A \ln (A / B)$ , with equality exactly when $a_{i} = (A / B) b_{i}$ at every coordinate.

Proof

1.1givenF1F2

If $p_{i} > 0 = q_{i}$ for some $i$ , [F1] gives $D (p ∥ q) = + \infty$ , so nonnegativity holds and equality to zero is impossible. Otherwise $p_{i} = 0$ whenever $q_{i} = 0$ ; apply [F2] to $a_{i} = p_{i}$ , $b_{i} = q_{i}$ . Both sums equal $1$ , and its left side is $D (p ∥ q)$ , so $D (p ∥ q) \geq 0$ .

2.1F1F2step 1.1algebra∎

In the absolutely continuous case, [F2] says equality holds exactly when $p_{i} = (1 / 1) q_{i} = q_{i}$ for every coordinate. Conversely, if $p = q$ , every positive-support term in [F1] is $p_{i} \ln 1 = 0$ , so $D (p ∥ q) = 0$ . Together with step 1.1 this proves the iff statement, including zero coordinates.

Depends on

Used by

Dependency tree · two levels

12 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Robert M. Gray, Entropy and Information Theory, corrected first edition (standard reference, not scraped)
Yury Polyanskiy and Yihong Wu, Information Theory: From Coding to Learning, author prepublication (standard reference, not scraped)