LemmaStatement: Literature-sourcedProof: AI-adaptedPipeline-generatedprecheck pending

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Mutual information splits into directed information and feedback

Statement

For finite sequences $X^{n}, Y^{n}$ , with directed information as in Finite directed information and causal conditioning, $I (X^{n}; Y^{n}) = I (X^{n} \to Y^{n}) + I (Y^{n - 1} \to X^{n}),$ where $I (Y^{n - 1} \to X^{n}) : = \sum_{i = 1}^{n} I (Y^{i - 1}; X_{i} ∣ X^{i - 1})$ and the $i = 1$ term is zero.

Facts & Assumptions

Given: A joint law of finite sequences $X^{n}, Y^{n}$ .

[F1]

Finite directed information and causal conditioning: $I (X^{n} \to Y^{n}) = \sum_{i} I (X^{i}; Y_{i} ∣ Y^{i - 1})$ .

[F2]

Finite entropy and mutual-information chain rules: finite entropy and mutual information obey the chain rules used to expand the sequence variables.

Proof

1.1F1F2algebra

Expand $I (X^{n}; Y^{n})$ by the chain rule in the order $Y_{1}, \dots, Y_{n}$ to obtain $\sum_{i = 1}^{n} I (X^{n}; Y_{i} ∣ Y^{i - 1})$ . For each $i$ , split $X^{n} = (X^{i}, X_{i + 1}^{n})$ and apply the chain rule again: $I (X^{n}; Y_{i} ∣ Y^{i - 1}) = I (X^{i}; Y_{i} ∣ Y^{i - 1}) + I (X_{i + 1}^{n}; Y_{i} ∣ X^{i}, Y^{i - 1}) .$ Summing the first terms gives $I (X^{n} \to Y^{n})$ by [F1].

2.1F1F2step 1.1algebra∎

Write $I (X^{n}; Y^{n}) = H (X^{n}) - H (X^{n} ∣ Y^{n})$ and each term of $I (X^{n} \to Y^{n})$ as $H (Y_{i} ∣ Y^{i - 1}) - H (Y_{i} ∣ Y^{i - 1}, X^{i})$ . The interleaved chain rule gives $H (X^{n} ∣ Y^{n}) + H (Y^{n}) = H (X^{n}, Y^{n}) = \sum_{i} [H (X_{i} ∣ X^{i - 1}, Y^{i - 1}) + H (Y_{i} ∣ Y^{i - 1}, X^{i})] .$ Subtracting the directed-information terms therefore yields $I (X^{n}; Y^{n}) - I (X^{n} \to Y^{n}) = H (X^{n}) - \sum_{i = 1}^{n} H (X_{i} ∣ X^{i - 1}, Y^{i - 1}) .$ The finite chain rule expands $H (X^{n}) = \sum_{i} H (X_{i} ∣ X^{i - 1})$ , so the right-hand side is $\sum_{i} I (X_{i}; Y^{i - 1} ∣ X^{i - 1}) = \sum_{i} I (Y^{i - 1}; X_{i} ∣ X^{i - 1})$ by symmetry of finite mutual information. This is the stated feedback term, including its zero $i = 1$ summand.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

9 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Abbas El Gamal and Young-Han Kim, Lecture Notes on Network Information Theory, arXiv:1001.3404v4 (standard reference, not scraped)