Alphabeta MathInformation Theory
Information theory studies quantitative limits on representing, transmitting, compressing, protecting, and extracting information from random objects. This draft track begins with finite probability laws, entropy, relative entropy, mutual information, and their measurable-space extensions. It then develops lossless source coding, independent and stationary ergodic sources, lossy compression and rate distortion, memoryless channels and their coding limits, finite-blocklength bounds, finite quantization and trellis/Viterbi algorithms, multiuser networks, secrecy, and quantum Shannon theory.
The track states every alphabet, sigma-algebra, probability law, channel kernel, support convention, logarithm base, error criterion, cost constraint, distortion measure, and regularity hypothesis at the point of use. Finite alphabet identities are kept separate from general measurable-space claims. Operational theorems distinguish achievability from converse and specify average or maximal error, block length, rate, and limiting regime. Network results distinguish exact capacity regions from inner and outer bounds; source and channel separation is stated only under proved hypotheses. Quantum pages reuse the repository's finite-dimensional state and channel interfaces and state support and trace conditions explicitly.
Pages remain research proposals until their local definitions, intermediate lemmas, proofs, algorithms, and actual prerequisite closures have been checked. Existing published library items are reused only within their stated scope. Source reading and citations provide provenance, not proof acceptance.
Pathway
Pages are grouped by how many dependency steps into this group they sit. Everything a page needs from this group appears above it.
Level 0
7 pagesThis finite-alphabet development defines probability vectors, stochastic kernels, support-aware f-divergences, KL divergence, and positive-order Rényi divergence using natural logarithms.
6 definitions, 5 lemmas, 5 theoremsExamples & counterexamples →Finite trellises turn finite-horizon path optimization into a layer-by-layer recurrence; the zero-length trellis has a unique empty path of score 1.
2 definitions, 2 algorithms, 2 lemmas, 2 theoremsExamples & counterexamples →- Finite-Alphabet Entropy and Divergence14 results
This page fixes the finite-alphabet conventions for natural-log nats and bits, conditional entropy, mutual information, and conditional mutual information.
4 definitions, 3 lemmas, 6 theorems, 1 corollaryExamples & counterexamples → For a finite alphabet, this page defines empirical types, type classes, and the finite product mass; it proves the product law is normalized with iid coordinates, then proves…
2 definitions, 7 lemmas, 2 theoremsExamples & counterexamples →- Quantum Information Measures21 results
This page defines finite-dimensional von Neumann entropy, support-sensitive quantum relative entropy, quantum mutual and conditional information, and classical-quantum…
4 definitions, 10 lemmas, 7 theoremsExamples & counterexamples → This page defines the finite two-party protocol output model, with a public authenticated transcript visible to Eve and a common abort outcome.
2 definitions, 4 lemmas, 1 theoremExamples & counterexamples →This page uses two-sided finite-alphabet path laws and the left shift for stationary sources, with the convention H 2(p)=H rm nat(p)/ln 2.
9 definitions, 1 algorithm, 9 lemmas, 9 theorems, 4 corollariesExamples & counterexamples →
Level 1
5 pages- Discrete Memoryless Channel Capacity18 results
This page begins with a general sequence of measurable channel kernels whose input and output spaces may vary by blocklength, then specializes to a fixed finite DMC with product transition law.
7 definitions, 7 lemmas, 4 theoremsExamples & counterexamples → For a finite alphabet and an IID law, bounded surprisal turns the finite-variance weak law into a quantitative information AEP.
4 definitions, 1 lemma, 1 theorem, 1 corollaryExamples & counterexamples →This page develops finite-alphabet conditional entropy for general standard-Borel observations, law entropy as self-mutual information, general mutual information as joint…
5 definitions, 3 lemmas, 8 theorems, 1 remarkExamples & counterexamples →- Prefix Codes and Finite Lossless Coding18 results
This page develops finite binary lossless coding for a finite full-support source.
5 definitions, 2 algorithms, 2 lemmas, 6 theorems, 3 corollariesExamples & counterexamples → This finite-dimensional development uses the support-sensitive quantum relative entropy with natural logarithms.
1 lemma, 2 theoremsExamples & counterexamples →
Level 2
6 pagesThis page develops finite-DMC capacity under a nonnegative letter cost, including the boundary budget equal to the minimum letter cost.
6 definitions, 15 lemmas, 3 theorems, 1 corollaryExamples & counterexamples →- Distributed Source Coding10 results
This page treats finite-alphabet IID lossless coding with separate encoders and one common decoder.
4 definitions, 5 lemmas, 1 theoremExamples & counterexamples → This page treats finite-alphabet, unconstrained, no-feedback discrete memoryless channels using natural logarithms and deterministic fixed-length encoders and decoders.
4 definitions, 8 lemmas, 9 theoremsExamples & counterexamples →This page proves two finite-dimensional asymptotic coding theorems.
3 definitions, 7 lemmas, 2 theoremsExamples & counterexamples →- Rate Distortion and Lossy Source Coding19 results
The page distinguishes joint-law mutual information from operational code rates, and expected distortion from excess-distortion probability.
6 definitions, 1 algorithm, 6 lemmas, 6 theoremsExamples & counterexamples → - Secrecy and Information Privacy14 results
All alphabets and probability laws on this page are finite, and information is measured in natural-log nats as in Information units and logarithm-base convention.
6 definitions, 4 lemmas, 4 theoremsExamples & counterexamples →
Level 3
4 pagesThis page develops measurable finite squared-error quantizers and exact finite empirical optimization.
3 definitions, 2 algorithms, 4 lemmas, 4 theoremsExamples & counterexamples →This page proves the finite-alphabet memoryless Wyner-Ziv theorem for expected normalized additive distortion, with natural logarithms and rates in nats.
3 definitions, 6 lemmas, 2 theoremsExamples & counterexamples →This page proves asymptotic separation for finite IID sources sent through finite discrete memoryless channels.
1 definition, 2 lemmas, 2 theoremsExamples & counterexamples →All registers and channels in this page are finite-dimensional, and all entropies use natural logarithms.
5 definitions, 20 lemmas, 7 theoremsExamples & counterexamples →
Level 4
1 page- Multiuser Channel Networks23 results
This page develops exact capacity results for a few carefully specified multiuser channel models and two feedback/source-channel inner bounds.
10 definitions, 6 lemmas, 7 theoremsExamples & counterexamples →
Level 5
1 page- Relay and Interference Channel Bounds36 results
This page separates three kinds of claims. For finite relay channels it proves a causal cut-set outer bound and a decode-forward inner bound; for the full-output-feedback…
7 definitions, 18 lemmas, 9 theorems, 2 corollariesExamples & counterexamples →