DefinitionDefinition: AI-adaptedProof: Not applicablePipeline-generatedjudge pass (gpt-6-luna)verified 2026-10-22 (gpt-6.1-sol)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Expected information gain

Definition

Assume the Axiom of Choice for the library's Kullback–Leibler divergence convention. For a design $d$ in a design-indexed dominated inverse experiment, let $G_{d} = {y : 0 < p_{d} (y) < \infty}$ be its positive finite evidence fibers. The expected information gain is $EIG (d) : = \int_{G_{d}} D (μ_{d}^{y} ∥ μ_{0}) p_{d} (y) ν (d y) \in [0, + \infty],$ where the integrand is defined to be zero outside $G_{d}$ . Here $D (P ∥ Q)$ is the directional Kullback–Leibler divergence of probability measures, with natural logarithms.

This integrand is measurable. Tonelli's theorem makes $p_{d}$ measurable, so $G_{d}$ is measurable. On $G_{d}$ , the likelihood ratio $r_{d} (θ, y) = p_{d} (y ∣ θ) / p_{d} (y)$ is the $μ_{0}$ -density of $μ_{d}^{y}$ . Set $r_{d} = 0$ off $G_{d}$ and wherever $p_{d} (y ∣ θ) = + \infty$ ; the latter set is $μ_{0}$ -null on each $y \in G_{d}$ because its integral is the finite value $p_{d} (y)$ . Thus $r_{d}$ is a finite-valued product-measurable function. With $ψ (0) = 0$ and $ψ (t) = t \log t$ for $t > 0$ , the KL definition gives $D (μ_{d}^{y} ∥ μ_{0}) = \int ψ (r_{d} (θ, y)) μ_{0} (d θ)$ . Tonelli makes the section integrals of $ψ (r_{d})^{+}$ and $ψ (r_{d})^{-}$ measurable; the negative integral is finite by the KL definition. Their difference is therefore measurable and extended-valued, and KL nonnegativity makes it $[0, + \infty]$ -valued. Setting it to zero off $G_{d}$ preserves measurability, so the displayed EIG integral is well-defined.

Definition references

The predictive density and posterior probability measure on positive finite evidence fibers.

The directional relative entropy is an extended integral with a finite negative part.

The general Radon–Nikodym convention used by the library KL definition assumes AC.

KL divergence between probability measures is nonnegative.

Tonelli makes the predictive density and the posterior KL integrand's section integrals measurable.

Chaloner and Verdinelli, §2.2

MacKay, §2

Remarks

The value $+ \infty$ is permitted. Before finite numerical optimization, either use the extended objective without subtracting entropies or impose and verify integrability for every design being compared. In particular, do not form a difference of two infinite prior and posterior entropies.

Remarks references

The library KL definition controls the negative part and defines its extended integral without an $\infty - \infty$ ambiguity.

Depends on

Used by

Finite-candidate information design Algorithm
Expected information gain under a wrong operator does not identify the operator-parameter pair Counterexample
Information gain on a singular prior support Example
Repeated measurements with correlated noise Example
Sensor placement along a high prior-variance direction Example
Expected information gain is joint relative entropy Theorem

Dependency tree · two levels

57 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Kathryn Chaloner and Isabella Verdinelli, Bayesian Experimental Design: A Review, Statistical Science 10(3) (1995), 273–304 (standard reference, not scraped)
David J. C. MacKay, Information-Based Objective Functions for Active Data Selection, Neural Computation 4(4) (1992), 590–604 (standard reference, not scraped)