Gaussian Orthogonal Decomposition and Quadratic Forms

Since $U^{T}U=I_n$ and $U$ is square, $U$ is invertible with inverse $U^T$, so $U{U}^T=I_n$. By [A2], Countable Choice holds. The continuous map $x\mapsto Ux$ is Borel measurable by [F5], so $(UZ{)}^{-1}(B)=Z^{-1}(U^{-1}(B))$ is measurable for every Borel $B\subseteq {\mathbb{R}}^n$. Thus $UZ$ is a Borel random vector and has a law by [F5]. For each $t\in {\mathbb{R}}^n$, [F1]–[F2] give $\mathbb{E}{e}^{i{t}^{T}UZ}=\exp (-\mathrm{\parallel }{U}^{T}t{\mathrm{\parallel }}^2\mathrm{/}2)=\exp (-\mathrm{\parallel }t{\mathrm{\parallel }}^2\mathrm{/}2)=\mathbb{E}{e}^{i{t}^{T}Z}$; the middle equality follows from $U{U}^T=I_n$.

By [A2] and [F3], the equality in step 1.1 at every $t$ is equality of the finite Borel Fourier transforms of the laws of $UZ$ and $Z$. Both are probability measures, hence have finite total variation; [F4] therefore gives $\mathcal{L}(UZ)=\mathcal{L}(Z)$.

By [F1], the coordinates of $Z$ are independent standard normals. Equality of joint laws in step 2.1 makes the coordinate law of $UZ$ the same product law, so its coordinates are again independent $N(0,1)$. This use of orthogonality depends on the Gaussian product law; no claim is made for arbitrary independent coordinates.

By [F1]–[F3], the joint law of $G$ is the finite product of the standard-normal coordinate laws. On every measurable rectangle, Tonelli and [F4] show that this product has density ${\prod }_{j=1}^k\varphi (z_j)=(2\pi {)}^{-k\mathrm{/}2}{e}^{-\mathrm{\parallel }z{\mathrm{\parallel }}^2\mathrm{/}2}$ relative to ${\lambda }_k$; uniqueness of the finite product law extends the identity from rectangles to every Borel set.

For $t>0$, [F1], step 1.1 and [F5] give $\mathbb{P}(Q_k\le t)=(2\pi {)}^{-k\mathrm{/}2}{\sigma }_{k-1}(S^{k-1}){\int }_0^{\sqrt{t}}{e}^{-r^2\mathrm{/}2}{r}^{k-1}dr=c_k{\int }_0^t{x}^{k\mathrm{/}2-1}{e}^{-x\mathrm{/}2}dx$ by the substitution $x=r^2$. The radial integral is finite near zero because $k\mathrm{/}2-1>-1$, and the polar measure is finite by [F5].

The random variable $Q_k=\mathrm{\parallel }G{\mathrm{\parallel }}^2$ is finite everywhere, so the events $\{Q_k\le m\}$ increase to the whole probability space as $m\to \mathrm{\infty }$. By [F6], their probabilities tend to $1$. If ${\sigma }_{k-1}(S^{k-1})$ were zero, step 2.1 would give zero probability for every finite $t$, a contradiction. Thus $c_k>0$. Step 2.1 then gives the density on $(0,\mathrm{\infty })$, and as $t\downarrow 0$ its integral tends to zero; hence $\mathbb{P}(Q_k=0)\le \mathbb{P}(Q_k\le t)\to 0$. Thus $Q_k>0$ almost surely.

The density in step 3.1 is locally integrable at zero and continuous and positive for $x>0$. Its integral over any interval $(a,b)\subset (0,\mathrm{\infty })$ is therefore positive, and its integral over a shrinking interval tends to zero. Thus the CDF is strictly increasing on $(0,\mathrm{\infty })$ and continuous at every positive $x$; step 2.1 also gives continuity at zero and no atom there. It is zero on $(-\mathrm{\infty },0]$.

Regard $P$ as a self-adjoint idempotent by [F2]. By [F1] it has an orthonormal eigenbasis. If $v$ is an eigenvector with eigenvalue $\lambda$, then $P^{2}v=Pv$ gives ${\lambda }^2=\lambda$, so $\lambda \in \{0,1\}$. Exactly $r$ eigenvalues are $1$, since the image of $P$ is spanned by precisely those eigenvectors. Reorder the basis so its first $r$ vectors have eigenvalue $1$, and let $U$ be the orthogonal matrix with these eigenvectors as columns. Then $P=U\mathrm{diag}(I_r,0)U^T$.

By [A2], the continuous-map measurability corollary applies. The maps $x\mapsto U^{T}x$, the two coordinate-block maps with zero-padding, and $x\mapsto Ux$ are continuous and therefore Borel measurable. Their compositions with the Borel random vector $Z$ are Borel random vectors; in particular, $W=U^{T}Z$, $PZ=U(W_1,\dots ,W_r,0,\dots ,0{)}^T$, and $(I_n-P)Z=U(0,\dots ,0,W_{r+1},\dots ,W_n{)}^T$ are random vectors. By [F3], $W_1,\dots ,W_n$ are independent standard normals. The first $r$ coordinates and the last $n-r$ coordinates generate independent sigma-algebras by [F5]; applying their measurable coordinate-block maps and then $U$ preserves independence. Thus $PZ$ and $(I_n-P)Z$ are independent.

Orthogonality of $U$ preserves Euclidean norm. Therefore $\mathrm{\parallel }PZ{\mathrm{\parallel }}^2={\sum }_{j=1}^r{W}_j^2$ and $\mathrm{\parallel }(I_n-P)Z{\mathrm{\parallel }}^2={\sum }_{j=r+1}^n{W}_j^2$. The squared-norm maps are continuous, hence Borel measurable by [A2]. By [F4] their laws are ${\chi }_r^2$ and ${\chi }_{n-r}^2$, and their independence follows from step 2.1 because measurable functions of independent random vectors remain independent by [F5]. When a block is empty its sum is $0$, so its law is ${\chi }_0^2={\delta }_0$; the constant zero vector is independent of every random vector by [F5].

Let $\mathrm{\Phi }$ be the standard-normal CDF. For $x<y$, [F1]–[F2] give $\mathrm{\Phi }(y)-\mathrm{\Phi }(x)={\int }_x^y\varphi (z)dz>0$. On every bounded interval $\varphi$ is bounded, so $\mathrm{\mid }\mathrm{\Phi }(y)-\mathrm{\Phi }(x)\mathrm{\mid }\le M\mathrm{\mid }y-x\mathrm{\mid }$ there and $\mathrm{\Phi }$ is continuous. The events $\{G\le m\}$ and $\{G>-m\}$ increase to the whole space as integer $m\to \mathrm{\infty }$; [F7] and total mass one give the CDF limits $1$ and $0$. Thus $\mathrm{\Phi }$ is continuous and strictly increasing on $\mathbb{R}$.

For $x\le 0$, $F_{r,s}(x)=0$ because its ratio is positive almost surely. For $x>0$, conditioning on $Q_s=q>0$ gives $F_{F_{r,s}}(x)={\int }_{(0,\mathrm{\infty })}{F}_{{\chi }_r^2}(xrq\mathrm{/}s)d{\chi }_s^2(q)$. If $x_j\to x>0$, continuity of the inner chi-square CDF and [F5] give continuity of this integral. If $0<x<y$, strict increase of the inner CDF on $(0,\mathrm{\infty })$ and positivity of $Q_s$ give a strict inequality between the two integrals. Dominated convergence as $x\downarrow 0$ and $x\to \mathrm{\infty }$ gives limits $0$ and $1$. Hence the F CDF is continuous on $\mathbb{R}$ and strictly increasing on $(0,\mathrm{\infty })$.

On the product space defining $t_k$, condition on $Q_k=q>0$. Independence and [F6] give $F_{t_k}(x)={\int }_{(0,\mathrm{\infty })}\mathrm{\Phi }(x\sqrt{q\mathrm{/}k})d{\chi }_k^2(q)$. If $x_j\to x$, the integrands converge pointwise and are bounded by $1$, so [F5] makes $F_{t_k}(x_j)\to F_{t_k}(x)$. If $x<y$, then $\mathrm{\Phi }(y\sqrt{q\mathrm{/}k})>\mathrm{\Phi }(x\sqrt{q\mathrm{/}k})$ for every $q>0$ by step 1.1, and [F3] says this is a full-measure set; integrating gives $F_{t_k}(y)>F_{t_k}(x)$. Dominated convergence as $x\to \pm \mathrm{\infty }$ gives limits $1$ and $0$.

The law $t_k$ is symmetric: replacing $G$ by $-G$ preserves the product law in [F4] and sends $T_k$ to $-T_k$. Continuity from step 2.1 therefore gives $F_{t_k}(0)=1\mathrm{/}2$ and $F_{t_k}(-x)=1-F_{t_k}(x)$. For either law and any $u\in (0,1)$, the CDF limits and continuity give points $a<b$ in its support with $F(a)<u<F(b)$; [F8] applied on $[a,b]$ gives a point with CDF value $u$, and strict increase gives uniqueness. These unique points are the stated quantiles. In particular, for $0<\alpha <1$, $q_{1-\alpha \mathrm{/}2}>0$ because $F_{t_k}(0)=1\mathrm{/}2<1-\alpha \mathrm{/}2$ and the CDF is strictly increasing.

By [A2], Countable Choice holds, so [F2] applies to the linear continuous map $x\mapsto \bar{x}$; hence it is a statistic on the stated Borel data space. Put $Z_i=(X_i-\mu )\mathrm{/}\sigma$. The affine pushforward definition in [F1] gives each $Z_i\sim N(0,1)$, and [F1] preserves their independence. Thus $Z=(Z_1,\dots ,Z_n)\sim N_n(0,I_n)$ by [F4].

Define $\bar{Z}=n^{-1}{\sum }_{i=1}^n{Z}_i$, $u=n^{-1\mathrm{/}2}(1,\dots ,1{)}^T$, and $P=u{u}^T$. Since $u^{T}u=1$, $P$ is a rank-one orthogonal projection. We have $u^{T}Z=\sqrt{n}\bar{Z}$ and $X-\bar{X}(1,\dots ,1{)}^T=\sigma (I_n-P)Z$. Therefore $\bar{X}=\mu +\sigma u^{T}Z\mathrm{/}\sqrt{n}$ and $S^2={\sigma }^2\mathrm{\parallel }(I_n-P)Z{\mathrm{\parallel }}^2\mathrm{/}(n-1)$.

By [F3], $PZ$ and $(I_n-P)Z$ are independent and $Q:=\mathrm{\parallel }(I_n-P)Z{\mathrm{\parallel }}^2\sim {\chi }_{n-1}^2$. The continuous linear map $z\mapsto u^{T}z$ is Borel by [A2] and [F2], and $u^{T}Z=u^{T}PZ$; the continuous squared-norm map on the complementary vector is likewise Borel. Thus $G:=u^{T}Z$ and $Q$ are measurable functions of the independent vectors in [F3], so they are independent by [F1]. By [F4], $G\sim N(0,1)$.

The formulas in step 2.1 express $\bar{X}$ as an affine function of $G$ and $S^2$ as ${\sigma }^{2}Q\mathrm{/}(n-1)$. Hence $\bar{X}$ and $S^2$ are independent, $(n-1)S^2\mathrm{/}{\sigma }^2=Q\sim {\chi }_{n-1}^2$, and $S>0$ almost surely by [F5].

The set $\{S>0\}$ is Borel and the displayed quotient is continuous there; assigning $0$ on $\{S=0\}$ therefore makes $\tilde{T}$ measurable. Since $S=0$ exactly when $Q=0$, step 2.1 gives $\tilde{T}=G\mathrm{/}\sqrt{Q\mathrm{/}(n-1)}$ on $\{Q>0\}$ and both versions are $0$ on $\{Q=0\}$. The variables $G$ and $Q$ are independent with laws $N(0,1)$ and ${\chi }_{n-1}^2$, so [F5] gives $\tilde{T}\sim t_{n-1}$.

Let $q=q_{n-1}(1-\alpha \mathrm{/}2)$. By [F5], the $t_{n-1}$ CDF is continuous and symmetric, so $F_{t_{n-1}}(0)=1\mathrm{/}2<1-\alpha \mathrm{/}2$ and strict increase gives $q>0$; symmetry then gives $F_{t_{n-1}}(-q)=\alpha \mathrm{/}2$. Step 5.1 therefore gives $\mathbb{P}(-q\le \tilde{T}\le q)=1-\alpha$. On the full-probability event $\{S>0\}$ from step 4.1, this event is exactly $\{\mu \text{ lies in the displayed interval}\}$ by multiplying by $S\mathrm{/}\sqrt{n}$ and rearranging. The events agree almost surely, so the interval has exact coverage $1-\alpha$.

By full column rank, the columns of $X$ are linearly independent. Apply [F2] to obtain orthonormal columns of $Q\in {\mathbb{R}}^{n\times p}$ with the same successive spans. Expressing each column of $X$ in this basis gives $X=QR$, where $R$ is upper triangular; its diagonal entries are the nonzero Gram–Schmidt residual norms, so $R$ is invertible. Let $W=\mathrm{col}X$. The columns of $X$ form a basis of $W$, so $\dim W=p$. By [F3], ${\mathbb{R}}^n=W\oplus W^{\perp }$. The subspace $W^{\perp }$ is finite-dimensional by [F10], and direct-sum dimension additivity in [F10] gives $n=p+\dim W^{\perp }$, hence $\dim W^{\perp }=n-p$. By [F3], choose an orthonormal basis of this complement with exactly $n-p$ vectors as the columns of $Q_0$; together with $Q$ it forms an orthonormal basis $[Q{Q}_0]$ of ${\mathbb{R}}^n$.

Set $U=[Q{Q}_0]$. Then $U$ is orthogonal. Standardizing the independent errors gives $\eta =\epsilon \mathrm{/}\sigma$ with independent standard-normal coordinates, so [F4] makes $U^T\eta =(Z_Q,Z_0)$ a standard-normal product vector with independent coordinate blocks. Since $X=QR$ and $X^{T}X=R^{T}R$, the normal-equation solution in [F1] is $\widehat{\beta }=R^{-1}{Q}^{T}Y=\beta +\sigma R^{-1}{Z}_Q$. Substituting $X=QR$ and $(R^{T}R{)}^{-1}=R^{-1}{R}^{-T}$ gives $P_X=QR(R^{T}R{)}^{-1}{R}^T{Q}^T=Q[R{R}^{-1}{R}^{-T}{R}^T]Q^T=Q{Q}^T$, because $R{R}^{-1}=I_p$ and $R^{-T}{R}^T=I_p$. Since $U$ is orthogonal, $I_n-P_X=Q_0{Q}_0^T$. Thus $(I_n-P_X)Y=\sigma Q_0{Z}_0$, so $RS{S}_X\mathrm{/}{\sigma }^2=\mathrm{\parallel }{Z}_0{\mathrm{\parallel }}^2\sim {\chi }_{n-p}^2$ by [F5]. By [A2] and [F7], the linear maps defining $\widehat{\beta }$ and $Q_0{Z}_0$, and the squared-norm map defining $RS{S}_X$, are Borel; hence these quantities are measurable functions of the independent coordinate blocks and $\widehat{\beta }$ is independent of $RS{S}_X$ by [F4]. The map defining $\widehat{\beta }$ is linear and Borel, so it is a statistic for the stated experiment. The residual map and squared norm are continuous, so $RS{S}_X$ and $S$ are continuous. The piecewise t version in the Statement is Borel: its quotient is continuous on the Borel set $S>0$ and its value on $S=0$ is constant. Its denominator has positive degree $n-p$, hence $S>0$ almost surely by [F5].

For $c\mathrm{\ne }0$, put $v=R^{-T}c$. Invertibility of $R$ gives $v\mathrm{\ne }0$. Complete $v\mathrm{/}\mathrm{\parallel }v\mathrm{\parallel }$ to an orthonormal basis of ${\mathbb{R}}^p$ by [F3]; applying [F4] to the resulting orthogonal matrix shows $G_c=v^T{Z}_Q\mathrm{/}\mathrm{\parallel }v\mathrm{\parallel }\sim N(0,1)$. Under [A2], the continuous linear map defining $G_c$ is Borel. It is independent of $Q_0:=\mathrm{\parallel }{Z}_0{\mathrm{\parallel }}^2\sim {\chi }_{n-p}^2$, because $Z_Q$ and $Z_0$ are independent blocks and $G_c$ and the squared-norm map are measurable functions of those blocks. Since $X^{T}X=R^{T}R$, $\mathrm{\parallel }v{\mathrm{\parallel }}^2=c^T{R}^{-1}{R}^{-T}c=c^T(X^{T}X{)}^{-1}c$. Thus on $Q_0>0$ the displayed contrast ratio is $G_c\mathrm{/}\sqrt{Q_0\mathrm{/}(n-p)}$; on $Q_0=0$ its version ${\tilde{T}}_c$ is zero. This is exactly the extension in the definition of $t_{n-p}$, so [F5]–[F6] give ${\tilde{T}}_c\sim t_{n-p}$.

For the nested models, let $W_D=W_U\cap W_R^{\perp }$. For $u\in W_U$, its orthogonal decomposition relative to $W_R$ has a $W_R$ component and a remaining component in $W_U\cap W_R^{\perp }=W_D$, because $W_R\subseteq W_U$. Thus [F3] gives $W_U=W_R\oplus W_D$ and ${\mathbb{R}}^n=W_U\oplus W_U^{\perp }$. All these subspaces are finite-dimensional by [F10]. Applying direct-sum dimension additivity in [F10] to the two decompositions gives $p=(p-q)+\dim W_D$ and $n=p+\dim W_U^{\perp }$, hence $\dim W_D=q$ and $\dim W_U^{\perp }=n-p$. Combining the orthogonal decompositions gives ${\mathbb{R}}^n=W_R\oplus W_D\oplus W_U^{\perp }$, including the zero-dimensional $W_R$ case $q=p$. Choose orthonormal bases in these three spaces and concatenate them into an orthogonal matrix $V$. By [F4], the coordinates of $V^T\epsilon \mathrm{/}\sigma$ in the three blocks are independent standard normals. Under the null, $P_R(X\beta )=P_U(X\beta )=X\beta$, so $(I-P_R)Y=(I-P_R)\epsilon =P_{W_D}\epsilon +P_{W_U^{\perp }}\epsilon$ and $(I-P_U)Y=(I-P_U)\epsilon =P_{W_U^{\perp }}\epsilon$: here [F8] identifies $I-P_R=P_{W_R^{\perp }}$ and $I-P_U=P_{W_U^{\perp }}$, while the direct sum in [F3] splits the first projection into the two orthogonal components. The coordinate-block maps and the linear projections are continuous, and squared norms are continuous, so [A2] and [F7] make all these random vectors and real variables measurable. Orthogonality and [F9] give $RS{S}_R-RS{S}_U={\sigma }^2{\sum }_{j=1}^q{G}_j^2$ and $RS{S}_U={\sigma }^2{\sum }_{j=1}^{n-p}{H}_j^2$, where the $G$ and $H$ coordinate blocks are independent standard-normal blocks. By [F5] these sums are independent ${\sigma }^2{\chi }_q^2$ and ${\sigma }^2{\chi }_{n-p}^2$ variables; both degrees are positive. The piecewise ratio $\tilde{F}$ is Borel, since it is continuous on the Borel set $RS{S}_U>0$ and constant on $RS{S}_U=0$. Its value is the scaled ratio on $Q_H:={\sum }_{j=1}^{n-p}{H}_j^2>0$ and $0$ when $Q_H=0$; by [F6] this is exactly the defined $F_{q,n-p}$ version. The F denominator is positive almost surely by [F5], so $\tilde{F}\sim F_{q,n-p}$.