TheoremStatement: AI-adaptedProof: AI-adaptedPipeline-generatedprecheck passjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Exact squared-error risk of an isotropic projection fit

Statement

Let $n, d \geq 1$ , let $X \in R^{n \times d}$ be fixed, and put $V = col (X)$ and $r = \dim V$ . Let $P_{V}$ be the orthogonal projection and let $μ \in R^{n}$ be fixed. Let $ε = (ε_{1}, \dots, ε_{n})$ be a centered random vector with square-integrable coordinates satisfying $E [ε_{i} ε_{j}] = σ^{2} 1_{i = j}$ for $σ^{2} \geq 0$ . For $Y = μ + ε$ and $\hat{μ} = {\hat{μ}}_{X} (Y)$ as in Fixed-design least squares as orthogonal projection,

$E ∥ \hat{μ} - μ ∥_{2}^{2} = ∥ P_{V} μ - μ ∥_{2}^{2} + r σ^{2} .$

In particular, when $μ \in V$ , the expected squared fitted-value error is exactly $r σ^{2}$ . The moment assumptions hold for centered Gaussian noise with covariance matrix $σ^{2} I_{n}$ ; Gaussianity is not otherwise needed.

Facts & Assumptions

Given: A fixed design matrix $X$ , its column space $V$ , the projection $P_{V}$ , a fixed $μ \in R^{n}$ , and the centered second-moment assumptions on $ε$ in the statement.

[F1]

The fixed-design fitted vector is the orthogonal projection of the observed vector onto the design column space (Fixed-design least squares as orthogonal projection).

[F2]

The projection is characterized by $P_{V} x \in V$ and $x - P_{V} x \in V^{⊥}$ (The orthogonal projection $P_{W} v$ is the $W$ -component in $V = W \oplus W^{⊥}$ ).

[F3]

An orthogonal projection is linear (Finite-dimensional orthogonal projections are linear and contractive).

[F4]

A finite-dimensional real inner-product space has an orthonormal basis, with the empty basis in dimension zero (Every finite-dimensional real or complex inner product space has an orthonormal basis).

[F5]

Orthogonal vectors satisfy Pythagoras (Pythagoras, the parallelogram identity, and the real and complex polarisation identities).

[F6]

For square-integrable real variables, variance and covariance are the expectations of centered products (Moments, variance, and covariance on a probability space).

[F7]

Expectation is additive on finite sums of integrable variables (Linearity, monotonicity, and the modulus bound for expectation).

[F8]

Risk is the expected loss of a decision rule under the sampling law (Loss functions and risk).

Proof

1.1F1F2F3algebra

Put $b = P_{V} μ - μ$ and $z = P_{V} ε$ . By [F2], $b \in V^{⊥}$ and $z \in V$ . Linearity [F3] gives $\hat{μ} - μ = b + z$ .

1.2F2F4algebra

Choose an orthonormal basis $ϕ_{1}, \dots, ϕ_{r}$ of $V$ by [F4]. The vector $\sum_{j = 1}^{r} ⟨ ε, ϕ_{j} ⟩ ϕ_{j}$ lies in $V$ and its residual from $ε$ is orthogonal to every basis vector, hence to $V$ ; uniqueness in [F2] identifies it with $P_{V} ε$ . Orthonormality therefore gives $∥ P_{V} ε ∥_{2}^{2} = \sum_{j = 1}^{r} ⟨ ε, ϕ_{j} ⟩^{2}$ , with the empty sum equal to zero when $r = 0$ .

1.3F4F6F7algebra

Write $ϕ_{j} = (ϕ_{1 j}, \dots, ϕ_{n j})$ . By [F6] and the given coordinate second moments, $E ⟨ ε, ϕ_{j} ⟩^{2} = \sum_{a, b = 1}^{n} ϕ_{a j} ϕ_{b j} E [ε_{a} ε_{b}] = σ^{2} \sum_{a = 1}^{n} ϕ_{a j}^{2} = σ^{2}$ for each $j$ . Summing with [F7] shows $E ∥ P_{V} ε ∥_{2}^{2} = r σ^{2}$ .

2.1step 1.1F5F7

The two summands in step 1.1 are orthogonal, so [F5] gives $∥ \hat{μ} - μ ∥_{2}^{2} = ∥ P_{V} μ - μ ∥_{2}^{2} + ∥ P_{V} ε ∥_{2}^{2}$ pointwise. Taking expectations and using [F7] separates the fixed first term from the noise term.

3.1step 2.1step 1.3F2F8∎

Substitute step 1.3 into step 2.1 to obtain the displayed identity. If $μ \in V$ , then $P_{V} μ = μ$ by [F2], giving the stated special case. By [F8], its left side is the risk under squared fitted-value loss.

Depends on

Used by

Collinear columns identify a fitted mean, not its coefficients Counterexample
Constrained least squares in a two-coordinate orthogonal sequence model Example
Zero training residual and positive test error Example

Dependency tree · two levels

28 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Philippe Rigollet and Jan-Christian Hütter, High-Dimensional Statistics: Lecture Notes, Chapter 2, Proposition 2.1 and Theorem 2.2 (standard reference, not scraped)
Sylvain Arlot, Model/estimator selection, Lecture on model selection, §§1–2 (standard reference, not scraped)