DefinitionDefinition: AI-adaptedProof: AI-generatedPipeline-generatedprecheck pending

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Finite-block information and entropy in bits

Definition

Consider a finite alphabet $A$ and a random block $X_{0}^{n - 1}$ with law $p_{n}$ on $A^{n}$ .

Define $\log_{2} t : = \ln t / \ln 2$ for $t > 0$ . The denominator is positive because $2 > 1$ , $\ln$ is strictly increasing, and $\ln 1 = 0$ by The natural logarithm as the inverse of the exponential function, Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm. For every positive-probability block $x$ , define its information in bits by $I_{n}^{(2)} (x) = - \log_{2} p_{n} (x)$ ; values on zero-probability blocks may be set to $+ \infty$ because they are never sampled.

Define block entropy in bits by $H_{n}^{(2)} : = E I_{n}^{(2)} (X_{0}^{n - 1}) = H_{n a t} (p_{n}) / \ln 2.$ Finite summation and Finite distributions, entropy, divergence, and stochastic matrices show this equals $- \sum_{x : p_{n} (x) > 0} p_{n} (x) \log_{2} p_{n} (x)$ . Thus the library's entropy interface stays in nats while this page's information and coding rates use bits. For any probability law $p$ on a finite set, put $H_{2} (p) : = H_{n a t} (p) / \ln 2 = - \sum_{x : p (x) > 0} p (x) \log_{2} p (x)$ , and write $H_{2} (U) : = H_{2} (P_{U})$ for a finite-valued random variable $U$ . For finite-valued $U, V$ , also define conditional entropy by $H_{2} (U ∣ V) = \sum_{v : P (V = v) > 0} P (V = v) H_{2} (P_{U ∣ V = v})$ ; zero-probability conditioning values are omitted.

Facts & Assumptions

Given: A finite alphabet $A$ and a random block $X_{0}^{n - 1}$ with law $p_{n}$ on $A^{n}$ .

[F1]

Finite distributions, entropy, divergence, and stochastic matrices: the existing finite entropy is $H_{n a t} (p) = - \sum_{x} p (x) \ln p (x)$ nats, with $0 \ln 0 = 0$ .

[F2]

The natural logarithm as the inverse of the exponential function and Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm: $\ln$ is the natural logarithm, is strictly increasing, and $\ln 1 = 0$ .

Proof

Given: The finite law and [F1]–[F2].

1.1F2givenalgebra

Define $\log_{2} t : = \ln t / \ln 2$ for $t > 0$ . The denominator is positive because $2 > 1$ , $\ln$ is strictly increasing, and $\ln 1 = 0$ by [F2]. For every positive-probability block $x$ , define its information in bits by $I_{n}^{(2)} (x) = - \log_{2} p_{n} (x)$ ; values on zero-probability blocks may be set to $+ \infty$ because they are never sampled.

2.1F1F2step 1.1algebra∎

Define block entropy in bits by $H_{n}^{(2)} : = E I_{n}^{(2)} (X_{0}^{n - 1}) = H_{n a t} (p_{n}) / \ln 2.$ Finite summation and [F1] show this equals $- \sum_{x : p_{n} (x) > 0} p_{n} (x) \log_{2} p_{n} (x)$ . Thus the library's entropy interface stays in nats while this page's information and coding rates use bits. For any probability law $p$ on a finite set, put $H_{2} (p) : = H_{n a t} (p) / \ln 2 = - \sum_{x : p (x) > 0} p (x) \log_{2} p (x)$ , and write $H_{2} (U) : = H_{2} (P_{U})$ for a finite-valued random variable $U$ . For finite-valued $U, V$ , also define conditional entropy by $H_{2} (U ∣ V) = \sum_{v : P (V = v) > 0} P (V = v) H_{2} (P_{U ∣ V = v})$ ; zero-probability conditioning values are omitted.

Depends on

Used by

Dependency tree · two levels

9 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Robert M. Gray, Entropy and Information Theory, corrected edition (standard reference, not scraped)