LemmaStatement: AI-adaptedProof: AI-adaptedPipeline-generatedprecheck passjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Exact probability of an empirical type class

Statement

Let $A$ be a finite alphabet, let $p$ be a probability vector on $A$ , and let $q \in P_{n} (A)$ . Under the product law $p^{\otimes n}$ , $p^{\otimes n} (T_{q}) = ∣ T_{q} ∣ \exp ⁣ (- n [H (q) + D (q ∥ p)]),$ where $H$ and $D$ are the finite entropy and relative entropy in Finite distributions, entropy, divergence, and stochastic matrices. The expression is $0$ when $q$ has positive mass outside $supp (p)$ , using $D (q ∥ p) = + \infty$ and $e^{- \infty} = 0$ .

Facts & Assumptions

Given: The finite alphabet $A$ , $n \geq 1$ , a product law $p^{\otimes n}$ , and an attainable type $q$ .

[F1]

Empirical types, n-types, and type classes: each $x^{n} \in T_{q}$ has $N (a ∣ x^{n}) = n q (a)$ , and $T_{q}$ contains exactly the words with that count vector.

[F2]

Finite distributions, entropy, divergence, and stochastic matrices: $H (q) = - \sum_{a} q (a) \ln q (a)$ with $0 \ln 0 = 0$ , and $D (q ∥ p) = \sum_{a : q (a) > 0} q (a) \ln (q (a) / p (a))$ , equal to $+ \infty$ when $q (a) > 0 = p (a)$ .

[F3]

Finite product mass on words: $p^{\otimes n} ({x^{n}}) = \prod_{i = 1}^{n} p (x_{i})$ , with event masses given by summing these atom weights.

[F4]

The finite product law has iid coordinate maps: this mass assignment is a probability law on $A^{n}$ .

Proof

1.1givenF1F2F3F4

If $q (a) > 0 = p (a)$ for some $a$ , every word in $T_{q}$ contains $n q (a) > 0$ copies of $a$ , so each word has product mass zero by [F3] and the event mass is zero by [F4]. The right-hand side is also zero because $D (q ∥ p) = + \infty$ .

2.1F1F2F3F4algebra∎

Otherwise $p (a) > 0$ whenever $q (a) > 0$ . Every $x^{n} \in T_{q}$ has mass $\prod_{a : q (a) > 0} p (a)^{n q (a)} = \exp (n \sum_{a : q (a) > 0} q (a) \ln p (a))$ by [F1, F3]. Expanding the finite sums in [F2] gives $H (q) + D (q ∥ p) = \sum_{a : q (a) > 0} q (a) \ln (1 / p (a))$ , so each word has mass $\exp (- n [H (q) + D (q ∥ p)])$ . Summing over the finite class proves the identity.

Depends on

Used by

Finite probability bounds for an empirical type Lemma

Dependency tree · two levels

11 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, second edition, Chapter 11 (standard reference, not scraped)