How statement and proof provenance work
The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.
- Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
- AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
- AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.
These labels describe origin, not correctness: citations and verification chips remain separate evidence.
Nonnegativity of finite relative entropy with a positive reference
Statement
Let and be probability vectors with for every . Then , with equality if and only if .
Facts & Assumptions
Given: Probability vectors on , with for every .
Finite distributions, entropy, divergence, and stochastic matrices: when no positive coordinate of is paired with a zero coordinate of .
Information units and logarithm-base convention: denotes the natural logarithm of the library's natural-logarithm definition.
Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm: The natural logarithm is continuous on .
The natural logarithm has derivative 1/x and equals the integral from 1 to x of 1/t: The natural logarithm is differentiable for positive inputs and has derivative .
The mean value theorem, as the case of Cauchy's: for continuous on with and differentiable on there is with : A continuous function on , differentiable on , has a point where its derivative equals its average rate of change.
for all , so the finite relative entropy in [F1] is defined by its displayed finite sum.
Proof
If , then . Otherwise let and . By [F2] and [F3], is continuous on ; by [F5], it is differentiable on with derivative . Applying [F6] gives with , so by [F4]. If , then ; if , then and . In either case . Thus , with equality only at .
Let . By [F1], [A1], and step 1.1 applied to for each , . If , both inequalities are equalities. Equality in the last requires outside , and [A1] then forces . Equality in the first requires each nonnegative gap from step 1.1, multiplied by , to vanish; hence for every . Conversely gives by [F1].
Depends on
- Finite distributions, entropy, divergence, and stochastic matrices
- Information units and logarithm-base convention
- Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm
- The natural logarithm has derivative 1/x and equals the integral from 1 to x of 1/t
- The mean value theorem, as the case $g(x) = x$ of Cauchy's: for $f$ continuous on $[a,b]$ with $a < b$ and differentiable on $(a,b)$ there is $c \in (a,b)$ with $f(b) - f(a) = f'(c)(b-a)$
Used by
Dependency tree · two levels
23 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.
Sources
- Stanford EE376A, Information Theory Course Notes, Chapter 4: Lossless Compression (standard reference, not scraped)
- MIT OpenCourseWare 6.450, Memory-less sources, prefix free codes, and entropy (standard reference, not scraped)