How statement and proof provenance work
The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.
- Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
- AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
- AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.
These labels describe origin, not correctness: citations and verification chips remain separate evidence.
Random coding achieves every rate below finite DMC capacity
Statement
For every finite DMC and every , there are fixed-length codes with and average error tending to zero.
Facts & Assumptions
Given: A finite DMC and a rate .
Finite DMC information capacity is attained: some input law satisfies .
Finite DMC information density: the information density is on the induced joint support, with its block value the coordinate sum.
IID DMC information density converges to mutual information: under IID input and DMC output, the normalized information density converges in probability to .
Finite stochastic kernels and The DMC product laws are stochastic kernels: each block transition law is a probability vector.
Fixed-length codes for a finite DMC and Average and maximal error of a fixed-length code: a selected codebook and deterministic decoder define a code and its average error.
Proof
If , one message gives zero error. Otherwise choose and . Then , so for all sufficiently large , . By [F3], the probability that the transmitted IID pair fails the threshold tends to zero.
Draw codewords independently from . Decode to the unique codeword with information density greater than , using a fixed default output if there is not exactly one. Conditional on the transmitted message, an error is contained in the union of the true-codeword miss event from step 1.1 and the event that some independent competing word passes the threshold. For one competitor independent of , [F2] gives by [F4], since every output reachable from a positive-probability input has positive mass. On a false pass, the nonnegative variable exceeds , so , and the false-pass probability is at most . A union bound over at most competitors makes this term at most .
The ensemble-averaged average error is at most the miss probability plus , so it tends to zero by step 1.1 and step 2.1. The ensemble has finite support; therefore at each some deterministic codebook has average error no greater than the ensemble mean. Its rate tends to , proving the claim.
Depends on
- Finite discrete memoryless channels
- The DMC product laws are stochastic kernels
- Fixed-length codes for a finite DMC
- Average and maximal error of a fixed-length code
- Single-letter information capacity of a finite DMC
- Finite DMC information capacity is attained
- Finite DMC information density
- IID DMC information density converges to mutual information
- Finite stochastic kernels
- Finite probability vectors
Used by
Dependency tree · two levels
18 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.
Sources
- Yury Polyanskiy and Yihong Wu, Information Theory: From Coding to Learning, Chapter 19 §19.2 (standard reference, not scraped)
- Robert Gallager, Principles of Digital Communication I, Chapter 8 §8.7.4, MIT OpenCourseWare (standard reference, not scraped)