TheoremStatement: Literature-sourcedProof: AI-adaptedPipeline-generatedprecheck pass

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

The full ordinary least-squares solution set and the unique ridge solution

Statement

Let $n, p \geq 1$ , $X \in R^{n \times p}$ , and $y \in R^{n}$ . The ordinary least-squares minimizers of $∥ y - X β ∥_{2}^{2}$ are exactly the solutions of $X^{T} X β = X^{T} y$ ; if $β_{0}$ is one such solution, the full minimizer set is $β_{0} + \ker X$ . It is a singleton if and only if the columns of $X$ are linearly independent. For every $λ > 0$ , the ridge objective $∥ y - X β ∥_{2}^{2} + λ ∥ β ∥_{2}^{2}$ has the unique minimizer ${\hat{β}}_{r i d g e} = (X^{T} X + λ I_{p})^{- 1} X^{T} y .$

Statement references

The Elements of Statistical Learning, Chapter 3 §§3.2 and 3.4

Rigollet and Hütter, §2.2, unconstrained least squares

Facts & Assumptions

Given: A finite real design matrix $X$ , a response vector $y$ , and, for the ridge claim, a real $λ > 0$ .

[F1]

Finite-dimensional least-squares minimizers exist, satisfy the normal equations, and form $β_{0} + \ker X$ (For a linear map $T : V \to W$ between finite-dimensional inner-product spaces, $x$ minimises $∥ T x - b ∥$ if and only if $T^{*} (T x - b) = 0$ , equivalently $T^{*} T x = T^{*} b$ ; minimisers exist and any two differ by an element of $\ker T$ ).

[F2]

A finite real matrix has a singular value decomposition with orthonormal left and right bases (Every linear map between finite-dimensional real or complex inner product spaces admits a singular value decomposition).

[F3]

The ridge objective is $∥ X β - y ∥_{2}^{2} + λ ∥ β ∥_{2}^{2}$ for $λ > 0$ (The Tikhonov regularised least-squares objective $∥ A x - b ∥_{2}^{2} + λ ∥ x ∥_{2}^{2}$ for $λ > 0$ ).

[F4]

That objective is strictly convex and has unique minimizer $(X^{T} X + λ I_{p})^{- 1} X^{T} y$ (For every $λ > 0$ , the Tikhonov objective is strictly convex and has the unique minimiser $(A^{*} A + λ I)^{- 1} A^{*} b$ ).

[F5]

The empirical squared-loss convention for OLS and the ridge objective is fixed in Fixed-design least-squares, ridge, lasso, group, and fused objectives.

Proof

1.1F1F5

Apply [F1] to the linear map $β \mapsto X β$ between finite-dimensional Euclidean spaces. It gives existence, the equation $X^{T} (X β - y) = 0$ , and the full set $β_{0} + \ker X$ .

2.1step 1.1algebra

The translate $β_{0} + \ker X$ has exactly one element if and only if $\ker X = {0}$ . By definition this is equivalent to linear independence of the columns, proving the OLS uniqueness assertion.

3.1F2F3F4F5algebra∎

By [F2], there is an orthonormal basis of $R^{p}$ in which $X^{T} X$ is diagonal with entries $σ_{1}^{2}, \dots, σ_{r}^{2}, 0, \dots, 0$ . Thus $X^{T} X + λ I_{p}$ has positive diagonal entries $σ_{j}^{2} + λ$ and $λ$ , so the inverse exists. The objective in [F3] is exactly the ridge objective in [F5], and [F4] gives its unique minimizer in the displayed formula.

Depends on

Used by

Dependency tree · two levels

15 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, 2nd ed., corrected 12th printing (January 2017) (standard reference, not scraped)
Philippe Rigollet and Jan-Christian Hütter, High-Dimensional Statistics: Lecture Notes, §2.2 (standard reference, not scraped)