How statement and proof provenance work
The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.
- Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
- AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
- AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.
These labels describe origin, not correctness: citations and verification chips remain separate evidence.
The deterministic Huffman code is prefix-optimal
Statement
For every finite full-support source, the algorithm Deterministic binary Huffman coding terminates, returns a binary prefix code, and minimizes among all binary prefix codes on the same alphabet. For a one-symbol source it returns the one-bit codeword , which is optimal among codes with nonempty codewords.
Facts & Assumptions
Given: A finite full-support source and the deterministic algorithm Deterministic binary Huffman coding.
Finite discrete sources and product blocks: The alphabet has symbols, every source weight is positive, and .
Binary prefix codes and their code trees: The code tree is the finite trie of codeword prefixes, and a prefix code has distinct codeword leaves with no codeword a proper prefix of another.
Finite binary source codes, extension, and expected length: Every codeword is nonempty and .
Binary Kraft criterion for finite positive codeword lengths: A finite positive integer length list with Kraft sum at most one is realized by a binary prefix code.
Deterministic binary Huffman coding: Order active nodes lexicographically by (weight, serial number), using the ordinary order on real weights and then integer serials.
Deterministic binary Huffman coding: Remove the first two nodes. Make a new parent whose left child is the first removed node and whose right child is the second, whose weight is their sum, and whose serial is the next unused integer.
Deterministic binary Huffman coding: If , return the one-bit codeword .
Proof
If , the algorithm returns by [F7]. Every codeword is nonempty by [F3], so every one-symbol code has expected length at least ; the returned code is prefix-free by [F2] and attains that value.
Suppose . Every merge removes two nodes and adds one, so the active count decreases by exactly one; after merges there is one root. Each original symbol remains one leaf, and the only new nodes are internal parents by [F6]. Distinct leaves cannot lie on one root-to-leaf path, so their path words are distinct and neither is a prefix of another. Every path has positive length because the root is internal. Thus [F2] gives a binary prefix code.
An optimal prefix code exists. For every integer , by induction: it holds at , and . Thus [F4] applied to the length list gives a finite prefix code, since its Kraft sum ; let its cost be . Any code with cost at most has for every , because and all summands in are nonnegative. Only finitely many positive integer length vectors satisfy these bounds, and at least one is realized. A least cost among those realized vectors exists and is a global minimum because every code outside the set has cost greater than .
For an active list of size two and total weight one, the algorithm merges its two nodes into a root with two leaves, giving lengths and cost . Every binary source code has nonempty words by [F3], so every competing prefix code has both lengths at least one and cost at least the sum of the weights, namely by [F1]. Thus the size-two case is optimal.
Suppose the active list has size and assume as induction hypothesis that every sequence of minimum-weight merges on a positive normalized list of size with distinct serials gives an optimal code.
In an optimal code tree, no internal vertex has exactly one child: contracting its sole edge preserves the leaves and their incomparability, so [F2] still gives a prefix code, while every codeword below the vertex loses one bit. At least one symbol is below it by the trie definition, and its positive weight by [F1] makes the expected length strictly smaller, a contradiction. Hence every internal vertex has two children. Choose a leaf of maximal depth . Its sibling cannot be internal, since that sibling would have a descendant leaf deeper than ; it is therefore another leaf at depth .
The first selected pair gives a reduced list of size with positive weights summing to one and distinct serials. Its subsequent merges are minimum-weight choices by [F5] and [F6]. The induction hypothesis in step 1.5 therefore makes the reduced tree optimal for its weights.
Let be the two symbols selected by the algorithm for its first merge; [F5] and [F6] make them two symbols of least weight. Place them at a sibling pair of leaves at depth without increasing cost. If is currently at depth and a symbol of weight occupies the chosen depth- leaf, swapping them changes the two cost terms from to , a difference . Then place in the sibling leaf by the same calculation, since its current occupant has weight at least . Relabeling leaves preserves the prefix property; the original code was optimal, so the resulting code is still optimal.
Contract the two sibling leaves labeled to one leaf with weight . The contracted tree is a prefix code on the reduced alphabet and its cost is the original cost minus , since both leaf depths decrease by one. Conversely, expanding any reduced prefix code by replacing the codeword for with its two children labeled preserves the prefix property and increases cost by . Therefore contraction sends an optimal original code to an optimal reduced code, and expanding an optimal reduced code gives an optimal original code.
By step 4.1, expansion of that reduced tree is optimal for the original weights. The algorithm terminates and returns a prefix code by step 1.2; the base case step 1.4 and the induction step prove its optimality for every .
Depends on
Used by
- Huffman expected length is within one bit of entropy Corollary
- Huffman tree for a dyadic three-symbol source Example
Cited to discharge well-definedness by Deterministic binary Huffman coding.
Dependency tree · two levels
7 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.
Sources
- Stanford EE376A, Information Theory Course Notes, Chapter 4: Lossless Compression (standard reference, not scraped)
- MIT OpenCourseWare 6.450, Memory-less sources, prefix free codes, and entropy (standard reference, not scraped)