Pipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Bayesian measurement design and information gain — Examples and Counterexamples

1 · Prerequisites

2 · Logical flowchart

3 · Definitions, theorems and proofs

None yet.

4 · Examples, counterexamples and false statements

CounterexampleConstruction: AI-generatedVerification: AI-generatedjudge pass (gpt-6-luna)verified 2026-10-22 (gpt-6.1-sol)Open item page →

Expected information gain under a wrong operator does not identify the operator-parameter pair

Statement refuted

The following implication is false: in every finite dominated inverse experiment, if expected information gain for a parameter is positive under a fixed nominal forward operator, then an unknown truth-operator pair is identifiable from the observation law, where identifiable means that distinct pairs induce distinct conditional observation laws.

Statement references

MacKay, §6.1 and Conclusion, model-correctness caveat

Chaloner and Verdinelli, §§1.2 and 2.2

Example

Take one scalar observation $Y = a θ + ϵ$ on $Y = {1, 2, 3}$ with counting measure and independent noise $ϵ$ uniform on ${0, 1}$ . In the actual model the unknown pair $H = (θ, a)$ is uniform on ${(1, 1), (2, \frac{1}{2})}$ . In the nominal model the operator is fixed at $a = 1$ and $θ$ is uniform on ${1, 2}$ . Under the actual model both pairs give the same law, uniform on ${1, 2}$ ; under the nominal model the expected information gain for $θ$ is $\frac{1}{2} \log 2 > 0$ .

Example references

A finite parameter space, counting observation measure, proper prior, and dominated likelihood define an indexed experiment.

EIG averages posterior-to-prior KL over the prior-predictive law.

MacKay, §6.1, misspecified-model example

Facts & Assumptions

Given: AC; the finite actual and nominal experiments in the Counterexample; the parameter priors and the counting measure on $Y$ .

[A1]

AC is inherited from the library KL convention. The Axiom of Choice

[F1]

Expected information gain equals $D (P_{θ, Y} ∥ P_{θ} \otimes P_{Y})$ for the specified dominated experiment. Expected information gain is joint relative entropy

[F2]

The expected information gain is the prior-predictive integral of posterior-to-prior KL divergence. Expected information gain

[F3]

KL divergence is nonnegative and vanishes exactly when the two probability measures agree. Kullback–Leibler divergence is nonnegative

[F4]

The natural logarithm is strictly increasing and $\log 1 = 0$ . Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm

[F5]

On a finite probability space, $D (P ∥ Q) = \sum_{x} P ({x}) \log (P ({x}) / Q ({x}))$ , with zero-mass terms equal to zero. Kullback–Leibler divergence

Verification

Given: The finite models in the Counterexample.

1.1A1F1F3givenalgebra

In the actual model, $a θ = 1$ for both possible pairs. Hence $P (Y \in {1, 2} ∣ H = (1, 1)) = P (Y \in {1, 2} ∣ H = (2, \frac{1}{2})) = 1$ , and both conditional laws assign probability $\frac{1}{2}$ to each of $1$ and $2$ . Thus $H$ and $Y$ are independent, so [F1] and [F3] give actual-pair EIG $D (P_{H, Y} ∥ P_{H} \otimes P_{Y}) = 0$ . The two distinct pairs have identical observation laws and are not identifiable.

1.2givenalgebra

In the nominal model, $θ = 1$ gives $Y \in {1, 2}$ equiprobably and $θ = 2$ gives $Y \in {2, 3}$ equiprobably. Therefore the predictive probabilities are $P (Y = 1) = \frac{1}{4}$ , $P (Y = 2) = \frac{1}{2}$ , and $P (Y = 3) = \frac{1}{4}$ .

2.1A1F2F4F5step 1.2algebra

The joint probabilities are $P (θ = 1, Y = 1) = \frac{1}{4}$ , $P (θ = 1, Y = 2) = \frac{1}{4}$ , $P (θ = 2, Y = 2) = \frac{1}{4}$ , and $P (θ = 2, Y = 3) = \frac{1}{4}$ . Dividing by the marginals in step 1.2 gives posterior $δ_{1}$ at $Y = 1$ , the original uniform prior at $Y = 2$ , and $δ_{2}$ at $Y = 3$ . By [F5], the corresponding KL values are $\log 2$ , $0$ , and $\log 2$ . Thus [F2] gives nominal EIG $\frac{1}{4} \log 2 + \frac{1}{2} \cdot 0 + \frac{1}{4} \log 2 = \frac{1}{2} \log 2 > 0$ by [F4].

3.1F1F3step 1.1step 2.1algebra∎

By [F1] and [F3] as applied in step 1.1, the actual-pair EIG is zero although the pairs are distinct and observationally identical; by step 2.1 the nominal fixed-operator EIG is positive. Thus the claimed implication fails.

ExampleConstruction: AI-generatedVerification: AI-generatedjudge pass (gpt-6-luna)verified 2026-10-22 (gpt-6.1-sol)Open item page →

Sensor placement along a high prior-variance direction

Example

Assume AC. Let $θ \sim N_{2} (0, C)$ with $C = diag (9, 1)$ . Compare two zero-cost one-sensor designs with independent unit-variance Gaussian noise: $d_{1}$ observes $A_{1} θ$ with $A_{1} = (1, 0)$ , and $d_{2}$ observes $A_{2} θ$ with $A_{2} = (0, 1)$ . Their expected information gains are $\frac{1}{2} \log 10$ and $\frac{1}{2} \log 2$ , respectively, so the sensor measuring the higher prior-variance direction is uniquely preferred.

Example references

Chaloner and Verdinelli, §2.2

MacKay, §3.1

Facts & Assumptions

Given: AC, the prior covariance $C = diag (9, 1)$ , the two row operators, and independent observation noise with covariance $1$ for both designs.

[A1]

AC is inherited from the EIG/KL convention. The Axiom of Choice

[F1]

For $C ⪰ 0$ and $Γ ≻ 0$ , $EIG = \frac{1}{2} \log \det (I + Γ^{- 1 / 2} A C A^{T} Γ^{- 1 / 2})$ . Linear Gaussian information gain has a log-determinant formula

[F2]

The natural logarithm is strictly increasing and $\log 1 = 0$ . Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm

Verification

Given: The hypotheses in the Example.

1.1A1F1givenalgebra

For $d_{1}$ , $A_{1} C A_{1}^{T} = 9$ and $Γ_{1} = 1$ , so [F1] gives $EIG (d_{1}) = \frac{1}{2} \log (1 + 9) = \frac{1}{2} \log 10$ . For $d_{2}$ , $A_{2} C A_{2}^{T} = 1$ and $Γ_{2} = 1$ , so [F1] gives $EIG (d_{2}) = \frac{1}{2} \log (1 + 1) = \frac{1}{2} \log 2$ .

2.1F2step 1.1algebra∎

The costs are equal and $10 > 2 > 0$ , so strict increase of the logarithm [F2] gives $EIG (d_{1}) > EIG (d_{2})$ . Thus $d_{1}$ is the unique maximizer among the two candidates.

ExampleConstruction: AI-generatedVerification: AI-generatedverified 2026-10-22 (gpt-6.1-sol)Open item page →

Repeated measurements with correlated noise

Example

Assume AC. Let $θ \sim N (0, 1)$ and observe $Y = A θ + η$ , where $A = (1, 1)^{T}$ and $η \sim N_{2} (0, Γ_{ρ})$ is independent of $θ$ , with $Γ_{ρ} = (\begin{array}{cc} 1 & ρ \\ ρ & 1 \end{array})$ and $- 1 < ρ < 1$ . The matrix $Γ_{ρ}$ is positive definite. The expected information gain is $\frac{1}{2} \log (1 + \frac{2}{1 + ρ})$ . In particular, correlation $ρ = \frac{1}{2}$ gives $\frac{1}{2} \log (7 / 3)$ , less than the independent-noise value $\frac{1}{2} \log 3$ at $ρ = 0$ .

Example references

Chaloner and Verdinelli, §2.2

MacKay, §3.1

Facts & Assumptions

Given: AC, $θ \sim N (0, 1)$ , $Y = A θ + η$ with $A = (1, 1)^{T}$ and independent $η \sim N_{2} (0, Γ_{ρ})$ , and $- 1 < ρ < 1$ .

[A1]

AC is inherited from the EIG/KL convention. The Axiom of Choice

[F1]

For independent $θ \sim N_{n} (m_{0}, C)$ and $η \sim N_{m} (0, Γ)$ with $C ⪰ 0$ and $Γ ≻ 0$ , observing $Y = A θ + η$ has the log-determinant EIG in the Statement of the cited theorem. Linear Gaussian information gain has a log-determinant formula

[F2]

The natural logarithm is strictly increasing. Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm

Verification

Given: The hypotheses in the Example.

1.1givenalgebra

The vectors $(1, 1)^{T}$ and $(1, - 1)^{T}$ are eigenvectors of $Γ_{ρ}$ with eigenvalues $1 + ρ$ and $1 - ρ$ . Both are positive for $- 1 < ρ < 1$ , so $Γ_{ρ} ≻ 0$ . With $C = 1$ , independence gives predictive covariance $S_{ρ} = A C A^{T} + Γ_{ρ} = (\begin{array}{cc} 2 & 1 + ρ \\ 1 + ρ & 2 \end{array})$ . Thus $\det S_{ρ} = 4 - (1 + ρ)^{2}$ and $\det Γ_{ρ} = 1 - ρ^{2}$ .

2.1A1F1step 1.1algebra

By [F1], $EIG (ρ) = \frac{1}{2} \log (\det S_{ρ} / \det Γ_{ρ}) = \frac{1}{2} \log (\frac{4 - (1 + ρ)^{2}}{1 - ρ^{2}})$ . Factoring numerator and denominator gives $\frac{4 - (1 + ρ)^{2}}{1 - ρ^{2}} = \frac{(3 + ρ) (1 - ρ)}{(1 - ρ) (1 + ρ)} = 1 + \frac{2}{1 + ρ}$ , proving the formula.

3.1F2step 2.1algebra∎

At $ρ = \frac{1}{2}$ , the ratio is $7 / 3$ ; at $ρ = 0$ , it is $3$ . Since $7 / 3 < 3$ , [F2] gives the strict EIG comparison in the Example.

ExampleConstruction: AI-generatedVerification: AI-generatedverified 2026-10-22 (gpt-6.1-sol)Open item page →

Information gain on a singular prior support

Example

Assume AC. Let $θ \sim N_{2} (0, C)$ with $C = diag (1, 0)$ , so its support is the first coordinate axis. Let $η \sim N (0, 1)$ be independent of $θ$ and compare the one-coordinate experiments $Y = A θ + η$ for $A_{⊥} = (0, 1)$ and $A_{∥} = (1, 0)$ . Their expected information gains are $0$ and $\frac{1}{2} \log 2$ , respectively. The log-determinant formula remains defined although $C$ is singular.

Example references

Clason, Chapter 13

Dashti and Stuart, §3.1

Facts & Assumptions

Given: AC, $θ \sim N_{2} (0, C)$ with $C = diag (1, 0)$ , $Y = A θ + η$ with independent $η \sim N (0, 1)$ , and the two row operators in the Example.

[A1]

AC is inherited from the finite Gaussian and EIG/KL interfaces. The Axiom of Choice

[F1]

The support of $N_{n} (m, C)$ is $m + Ran (C)$ , including singular $C$ . Support and pseudoinverse coordinates of a finite singular Gaussian

[F2]

For independent $θ \sim N_{n} (m_{0}, C)$ and $η \sim N_{m} (0, Γ)$ with $C ⪰ 0$ and $Γ ≻ 0$ , the linear-Gaussian EIG formula applies to $Y = A θ + η$ , including singular $C$ . Linear Gaussian information gain has a log-determinant formula

[F3]

The natural logarithm is strictly increasing and $\log 1 = 0$ . Order, continuity, range, and the product, quotient, and reciprocal laws for the natural logarithm

Verification

Given: The hypotheses in the Example.

1.1A1F1givenalgebra

The range of $C$ is $span {(1, 0)^{T}}$ , so [F1] gives the stated support. Also $A_{⊥} C A_{⊥}^{T} = 0$ and $A_{∥} C A_{∥}^{T} = 1$ .

2.1A1F2step 1.1algebra

Applying [F2] with noise covariance $1$ gives $EIG (A_{⊥}) = \frac{1}{2} \log (1 + 0) = 0$ and $EIG (A_{∥}) = \frac{1}{2} \log (1 + 1) = \frac{1}{2} \log 2$ .

3.1F3step 2.1algebra∎

Since $2 > 1$ , [F3] gives $\log 2 > \log 1 = 0$ . The second coordinate therefore contributes no information under the prior, while the first-coordinate measurement has strictly positive EIG. No inverse or determinant of $C$ is taken.

Bayesian measurement design and information gain — Examples and Counterexamples

2 · Logical flowchart

3 · Definitions, theorems and proofs

4 · Examples, counterexamples and false statements

Expected information gain under a wrong operator does not identify the operator-parameter pair

Statement refuted

Statement references

Example

Example references

Facts & Assumptions

Verification

Sensor placement along a high prior-variance direction

Example

Example references

Facts & Assumptions

Verification

Repeated measurements with correlated noise

Example

Example references

Facts & Assumptions

Verification

Information gain on a singular prior support

Example

Example references

Facts & Assumptions

Verification

Sources