TheoremStatement: Literature-sourcedProof: AI-adaptedPipeline-generatedprecheck passjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Least-squares fitted vectors and residuals are unique

Statement

Let $n, p \geq 1$ , $X \in R^{n \times p}$ , $y \in R^{n}$ , and $C = col (X) \subseteq R^{n}$ . For the least-squares solution set $B_{L S} (y)$ of Least-squares solution set, the following hold:

$B_{L S} (y)$ is nonempty, and $b$ is in it exactly when $X^{T} (X b - y) = 0$ .
If $b_{0} \in B_{L S} (y)$ , then $B_{L S} (y) = b_{0} + \ker X$ .
Every least-squares solution has the same fitted vector $P_{C} y$ and residual $y - P_{C} y$ .
The direction space $\ker X$ of the affine solution set has dimension $p - rank (X)$ .

The theorem makes no statistical moment or distributional assumption.

Facts & Assumptions

Given: Integers $n, p \geq 1$ , a real matrix $X \in R^{n \times p}$ , and $y \in R^{n}$ . Put $T_{X} : R^{p} \to R^{n}$ , $T_{X} b : = X b$ , and $C = col (X) = im T_{X}$ .

[F1]

The least-squares solution set is $B_{L S} (y) = {arg min}_{b \in R^{p}} ∥ y - X b ∥_{2}^{2}$ ; it is set-valued until uniqueness is proved. Least-squares solution set.

[F2]

For a finite-dimensional linear map $T$ and target $y$ , minimizers of $∥ T b - y ∥$ exist, are characterized by $T^{*} (T b - y) = 0$ , and form $b_{0} + \ker T$ . For a linear map $T : V \to W$ between finite-dimensional inner-product spaces, $x$ minimises $∥ T x - b ∥$ if and only if $T^{*} (T x - b) = 0$ , equivalently $T^{*} T x = T^{*} b$ ; minimisers exist and any two differ by an element of $\ker T$ .

[F3]

The Euclidean inner product on $R^{m}$ is the dot product $⟨ x, y ⟩ = \sum_{k < m} x_{k} y_{k}$ . The Euclidean inner product $⟨ x, y ⟩ = \sum_{k < n} x_{k} y_{k}$ on $R^{n}$ .

[F4]

In standard real coordinate bases, the adjoint matrix of a linear map is its transpose. In orthonormal bases, the matrix of the adjoint is the conjugate transpose of the matrix.

[F5]

The orthogonal projection $P_{C} y$ is the $C$ -component of $y$ and $y - P_{C} y \in C^{⊥}$ . The orthogonal projection $P_{W} v$ is the $W$ -component in $V = W \oplus W^{⊥}$ .

[F6]

$P_{C} y$ is the unique point of $C$ nearest to $y$ . The orthogonal projection is the unique nearest point in the subspace.

[F7]

For a linear map with finite-dimensional domain, $\dim V = \dim \ker T + \dim im T$ . Rank-nullity: $\dim_{F} V = nullity T + rank T$ .

[F8]

Squaring is strictly increasing on nonnegative reals: $s < t$ if and only if $s^{2} < t^{2}$ for $s, t \geq 0$ . Squaring is monotone on the nonnegatives.

[F9]

The standard coordinate vectors form a basis of $R^{p}$ , so $\dim R^{p} = p$ . The standard list $e : n \to F^{n}$ with $e_{i} (i) = 1_{F}$ and $e_{i} (j) = 0_{F}$ for $j \neq i$ is an ordered basis of $F^{n}$ ; hence $\dim_{F} F^{n} = n$ , and $F^{0}$ is the zero space with basis $\emptyset$ and dimension $0$ .

[F10]

The matrix rank of $X$ equals the rank of $T_{X}$ , which is $\dim im T_{X}$ . The rank of a matrix equals the rank of the linear map $x \mapsto A x$ .

Proof

technique · direct

1.1F1F2F3F4F8algebra

Regard $X$ as the linear map $T_{X} : R^{p} \to R^{n}$ , $T_{X} b = X b$ , with the Euclidean inner products of [F3]. Every residual norm is nonnegative, and [F8] says squaring is strictly increasing on the nonnegative reals. Therefore the minimizers of $∥ y - X b ∥_{2}^{2}$ are exactly the minimizers of $∥ X b - y ∥_{2}$ . Applying [F2] and identifying $T_{X}^{*} = X^{T}$ by [F4] gives a minimizer, the normal equation $X^{T} (X b - y) = 0$ , and the full solution set $b_{0} + \ker X$ .

1.2F2F5F6choose

The image of $T_{X}$ is $C$ . Every minimizer maps to a point of $C$ nearest to $y$ , and every point of $C$ has a preimage under $T_{X}$ . By [F5] and [F6] that nearest point is exactly $P_{C} y$ and is unique. Thus every minimizing $b$ has fitted vector $X b = P_{C} y$ , and its residual is the unique vector $y - P_{C} y$ .

2.1F7F9F10step 1.1algebra∎

By step 1.1 the solution set is the translate $b_{0} + \ker X$ , whose direction space is $\ker X$ . By rank-nullity [F7] and $\dim R^{p} = p$ from [F9], $\dim \ker X = p - \dim im T_{X}$ . Fact [F10] identifies $\dim im T_{X} = rank (X)$ , giving the claimed direction-space dimension.

Depends on

Used by

Cited to discharge well-definedness by Least-squares solution set.

Dependency tree · two levels

59 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Statistics for Applications, Section 15: Multiple Linear Regression (standard reference, not scraped)
Statistics: Lecture Three — Linear Models (standard reference, not scraped)