Full-rank OLS is unbiased with its classical covariance

Given: Integers $n,p\ge 1$, a fixed-design first-moment regression model, full-column-rank $X\in {\mathbb{R}}^{n\times p}$, and the finite second-moment assumptions in the Statement.

The fixed design has response mean ${\mathbb{E}}_{\beta }Y=X\beta$. Fixed-design linear regression model.

Least-squares minimizers satisfy $X^T(Xb-Y)=0$; when $\ker X=\{0\}$ the minimizer is unique. Least-squares fitted vectors and residuals are unique.

For the endomorphism $G(v)=X^{T}Xv$ of ${\mathbb{R}}^p$, rank-nullity gives $\dim {\mathbb{R}}^p=\dim \ker G+\dim \mathrm{im}G$. Rank-nullity: ${\dim }_{F}V=\mathrm{nullity}T+\mathrm{rank}T$.

Expectation is linear on finite sums of integrable random variables. Linearity, monotonicity, and the modulus bound for expectation.

Covariance is bilinear in finite linear combinations of square-integrable random variables. Covariance is symmetric and bilinear in finite linear combinations.

Variance and covariance are expectations of centered products. Moments, variance, and covariance on a probability space.

The standard coordinate vectors form a basis of ${\mathbb{R}}^p$, so $\dim {\mathbb{R}}^p=p$. The standard list $e:n\to F^n$ with $e_i(i)=1_F$ and $e_i(j)=0_F$ for $j\mathrm{\ne }i$ is an ordered basis of $F^n$; hence ${\dim }_F{F}^n=n$, and $F^0$ is the zero space with basis $\mathrm{\varnothing }$ and dimension $0$.

The Euclidean inner product is positive definite and $\mathrm{\parallel }z{\mathrm{\parallel }}_2^2=z^{T}z$; in particular $z\mathrm{\ne }0$ implies $\mathrm{\parallel }z{\mathrm{\parallel }}_2^2>0$. The Euclidean inner product $⟨x,y⟩={\sum }_{k<n}{x}_k{y}_k$ on ${\mathbb{R}}^n$.

The error coordinates are square-integrable, have mean zero, and satisfy ${\mathrm{Cov}}_{\beta }({ϵ}_{\beta })={\sigma }^2{I}_n$ for ${\sigma }^2>0$.