TheoremStatement: Literature-sourcedProof: AI-adaptedPipeline-generatedjudge pass (gpt-6-luna)verified 2026-10-22 (gpt-6.1-sol)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Linear Gaussian information gain has a log-determinant formula

Statement

Assume the Axiom of Choice. Let $n, m \geq 1$ , let $m_{0} \in R^{n}$ , let $C \in R^{n \times n}$ be symmetric positive semidefinite, let $A \in R^{m \times n}$ , and let $Γ \in R^{m \times m}$ be symmetric positive definite. Suppose $θ \sim N_{n} (m_{0}, C)$ and $η \sim N_{m} (0, Γ)$ are independent, and observe $Y = A θ + η$ . With $S = A C A^{T} + Γ$ , the expected information gain of this experiment is $EIG = \frac{1}{2} \log \det ⁣ (I_{m} + Γ^{- 1 / 2} A C A^{T} Γ^{- 1 / 2}) .$ Here $Γ^{- 1 / 2}$ is the inverse of the unique symmetric positive-definite square root of $Γ$ , and $\log$ is the natural logarithm. The matrix inside the log determinant is positive definite. This formula permits singular $C$ and never uses differential entropy of $θ$ ; the mean $m_{0}$ cancels. Noise depending on $θ$ is outside the hypotheses and need not satisfy the formula.

Statement references

Expected information gain is the joint KL divergence of parameter and observation from the product of their marginals.

For singular prior covariance and positive-definite observation noise, the finite linear Gaussian posterior exists and only the observation covariance is inverted.

A finite multivariate Gaussian law may have singular covariance.

Positive-definite Gaussian covariance gives the stated Lebesgue density.

Logarithms use the natural logarithm.

Chaloner and Verdinelli, §2.2

MacKay, §3.1

Rasmussen and Williams, Chapter 2

Facts & Assumptions

Given: The hypotheses in the Statement, with the expected information gain defined for the dominated conditional Gaussian likelihood and its proper Gaussian prior.

[A1]

AC is the explicit hypothesis of the finite Gaussian posterior interface and the library KL convention. The Axiom of Choice

[F1]

The joint-KL identity gives $EIG = D (P_{θ, Y} ∥ P_{θ} \otimes P_{Y})$ under the dominated model. Expected information gain is joint relative entropy

[F2]

For $θ \sim N_{n} (m_{0}, C)$ and independent $η \sim N_{m} (0, Γ)$ with $C ⪰ 0$ and $Γ ≻ 0$ , $S = A C A^{T} + Γ$ is positive definite and the posterior formula uses only $S^{- 1}$ ; singular $C$ is permitted. Finite linear Gaussian inverse posterior with singular prior covariance

[F3]

A finite independent stack of possibly singular Gaussian vectors is jointly Gaussian, and linear projections of a multivariate normal have the corresponding projected mean and covariance. A finite stack of independent Gaussian vectors is jointly Gaussian Multivariate normal law, including singular covariance

[F4]

A positive-definite multivariate normal with covariance $V$ has density $(2 π)^{- m / 2} \det (V)^{- 1 / 2} \exp (- \frac{1}{2} (y - a)^{T} V^{- 1} (y - a))$ . A positive-definite multivariate normal law has its Gaussian density

[F5]

A positive-definite symmetric matrix has a unique positive-definite square root; determinants are multiplicative. A non-negative operator has a unique non-negative square root For same-sized finite square matrices over a commutative ring, $\det (A B) = \det (A) \det (B)$

Proof

Given: The hypotheses in the Statement.

1.1givenF2algebra

For every nonzero $v \in R^{m}$ , $v^{T} S v = (A^{T} v)^{T} C (A^{T} v) + v^{T} Γ v > 0$ because $C ⪰ 0$ and $Γ ≻ 0$ . Thus $S ≻ 0$ and both observation covariances have positive determinants. The same conclusion is recorded in [F2].

2.1A1F3F4step 1.1algebra

By [F3], $(θ, η)$ is jointly Gaussian, so its linear image $Y = A θ + η$ is Gaussian with mean $A m_{0}$ and covariance $S$ . Conditional on $θ$ , $Y$ is a translate of $η$ and has covariance $Γ$ . Since both $S$ and $Γ$ are positive definite, [F4] gives the positive conditional and predictive densities needed in [F1].

3.1F1F4step 1.1step 2.1algebra

Write $z = Y - A m_{0}$ and $e = Y - A θ = η$ . By [F4], the log likelihood ratio in [F1] is $\log \frac{p (Y ∣ θ)}{p_{Y} (Y)} = \frac{1}{2} \log \frac{\det S}{\det Γ} - \frac{1}{2} e^{T} Γ^{- 1} e + \frac{1}{2} z^{T} S^{- 1} z$ . The two quadratic terms have finite expectations: expanding each finite sum and using $E [e e^{T}] = Γ$ and $E [z z^{T}] = S$ gives $E [e^{T} Γ^{- 1} e] = tr (Γ^{- 1} Γ) = m$ and $E [z^{T} S^{- 1} z] = tr (S^{- 1} S) = m$ . Their expectations cancel, so [F1] yields $EIG = \frac{1}{2} \log (\det S / \det Γ)$ .

4.1F5step 3.1algebra∎

Let $W = Γ^{- 1 / 2}$ . Since $Γ^{1 / 2} W = I_{m}$ , matrix multiplication gives $S = Γ^{1 / 2} (I_{m} + W A C A^{T} W) Γ^{1 / 2}$ . The middle matrix is positive definite because $A C A^{T} ⪰ 0$ . By [F5], $\det S = \det (Γ^{1 / 2})^{2} \det (I_{m} + W A C A^{T} W) = \det Γ \det (I_{m} + W A C A^{T} W)$ . Substitution in step 3.1 proves the formula. No inverse or determinant of $C$ was used, so singular $C$ is included; centering by $m_{0}$ in $z$ proves the mean cancellation.

Depends on

Used by

Dependency tree · two levels

67 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Kathryn Chaloner and Isabella Verdinelli, Bayesian Experimental Design: A Review, Statistical Science 10(3) (1995), 273–304 (standard reference, not scraped)
David J. C. MacKay, Information-Based Objective Functions for Active Data Selection, Neural Computation 4(4) (1992), 590–604 (standard reference, not scraped)
Carl Edward Rasmussen and Christopher K. I. Williams, Gaussian Processes for Machine Learning, MIT Press (2006) (standard reference, not scraped)