LemmaStatement: Literature-sourcedProof: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Finite product KL is the sum of coordinate KLs

Statement

Let $r \geq 1$ , let $S_{1}, \dots, S_{r}$ be finite nonempty sets, and let $u_{k}, v_{k}$ be probability vectors on $S_{k}$ . Put $U = ⨂_{k = 1}^{r} u_{k}$ and $V = ⨂_{k = 1}^{r} v_{k}$ on $S_{1} \times \dots \times S_{r}$ . With $D$ the finite directional KL divergence, zero source masses contributing zero and positive source mass against zero reference mass contributing $+ \infty$ , one has $D (U ∥ V) = \sum_{k = 1}^{r} D (u_{k} ∥ v_{k}),$ with equality also when one or both sides are $+ \infty$ .

Statement references

Finite Kullback–Leibler divergence: $D (P ∥ Q) = \sum_{x : p_{x} > 0} p_{x} \log (p_{x} / q_{x})$ , with $+ \infty$ for positive source mass where the reference mass is zero.

Facts & Assumptions

Given: A finite list of finite nonempty sets and probability vectors $u_{k}, v_{k}$ . Prove additivity of the finite KL of their product laws, including singular support.

[F1]

Finite Kullback–Leibler divergence: the finite KL convention is directional; zero source terms vanish and positive source mass over zero reference mass gives $+ \infty$ .

Proof

1.1givenF1construct

Suppose some coordinate $k$ has $u_{k} (a_{k}) > 0 = v_{k} (a_{k})$ . For every other coordinate $i$ , the finite probability vector $u_{i}$ has a nonempty support set $A_{i} = {a : u_{i} (a) > 0}$ . Choose one point from each of the finitely many $A_{i}$ by recursion over the finite index list; this uses only finite choice. Then $U (a_{1}, \dots, a_{r}) > 0$ while $V (a_{1}, \dots, a_{r}) = 0$ . By [F1], $D (U ∥ V) = + \infty$ and $D (u_{k} ∥ v_{k}) = + \infty$ , so both sides of the claimed identity are $+ \infty$ .

1.2givenF1algebra

Otherwise, for every $i$ and every $a_{i}$ with $u_{i} (a_{i}) > 0$ , $v_{i} (a_{i}) > 0$ . For each tuple with $U (a) > 0$ , $\log \frac{U (a)}{V (a)} = \sum_{i = 1}^{r} \log \frac{u_{i} (a_{i})}{v_{i} (a_{i})} .$ Substitute this finite sum into the definition [F1], interchange the finite sums, and sum out every coordinate except $i$ ; each such marginal sum is $\sum_{a_{k}} u_{k} (a_{k}) = 1$ . The result is $D (U ∥ V) = \sum_{i} \sum_{a_{i} : u_{i} (a_{i}) > 0} u_{i} (a_{i}) \log (u_{i} (a_{i}) / v_{i} (a_{i})) = \sum_{i} D (u_{i} ∥ v_{i})$ .

2.1F1step 1.1step 1.2∎

Steps 1.1 and 1.2 cover all possibilities, including $r = 1$ and coordinates with zero source mass. This proves the extended-valued identity.

Depends on

Finite Kullback–Leibler divergence

Used by

Masked-diffusion KL reduces to weighted cross-entropy Theorem

Dependency tree · two levels

5 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Yury Polyanskiy and Yihong Wu, Information Theory: From Coding to Learning (standard reference, not scraped)