Multivariate normal, Wishart, and Hotelling $T^2$

For $k=0$, the first column is nonzero almost surely: the event $G_1=0$ is contained in the event that its first coordinate is zero, which has probability zero because a standard normal has a density. For $1\le k<p$, suppose the first $k$ columns are linearly independent. For each fixed such tuple $v_1,\dots ,v_k$, their span $V$ is proper; by [F3] choose $a\mathrm{\ne }0$ with $a\perp V$. The next column is independent of this tuple by [F1], and [F2] gives $a^T{G}_{k+1}\sim N(0,\mathrm{\parallel }a{\mathrm{\parallel }}^2)$, so $\mathbb{P}(G_{k+1}\in V)=0$ because that event is contained in $\{a^T{G}_{k+1}=0\}$. The set of dependent $(k+1)$-tuples is Borel, since dependence is the simultaneous vanishing of its $(k+1)\times (k+1)$ minors and [F5] makes these determinant maps Borel. Integrating these zero-probability sections by [F1] and [F4], and adding the null set of dependent first $k$-tuples, shows that the first $k+1$ columns are linearly independent almost surely.

Applying step 1.1 successively through $k=p-1$ shows that the square matrix $B=[G_1\mathrm{\mid }\cdots \mathrm{\mid }{G}_p]$ is invertible almost surely. For any nonzero $v\in {\mathbb{R}}^p$, the first $p$ coordinates of $G^{T}v$ are $B^{T}v\mathrm{\ne }0$, so $v^{T}G{G}^{T}v=\mathrm{\parallel }{G}^{T}v{\mathrm{\parallel }}^2>0$. Thus $G{G}^T$ is positive definite almost surely.

By [F1], realize $W=G{G}^T$ with $G=[g_1\mathrm{\mid }\cdots \mathrm{\mid }{g}_m]$ and independent $g_j\sim N_p(0,I_p)$. For any square matrix $A$, extend inversion by setting $J(A)=A^{-1}$ when $\det A\mathrm{\ne }0$ and $J(A)=0$ otherwise. This is Borel: the invertible matrices form an open set, inversion is continuous there, and the complementary branch is constant; [F11] supplies Euclidean Borel measurability. Thus $J(G{G}^T)$ is a Borel matrix-valued random element and agrees with $W^{-1}$ whenever $G{G}^T$ is invertible. Since $m\ge p$, [F2] makes $G$ full row rank almost surely; hence for nonzero $v$, $v^{T}Wv=\mathrm{\parallel }{G}^{T}v{\mathrm{\parallel }}^2>0$ and $W$ is invertible almost surely.

For $p\ge 2$, write the first row of $G$ as $g$ and the remaining $p-1$ rows as $G_2$. The event $E_2=\{\det (G_2{G}_2^T)\mathrm{\ne }0\}$ is Borel. Define $C^{†}=(G_2{G}_2^T{)}^{-1}$ on $E_2$ and $C^{†}=0$ on $E_2^c$, and put $P=G_2^T{C}^{†}{G}_2$; this is Borel by the same piecewise-inversion argument as in step 1.1. By [F2], $E_2$ has probability one. On $E_2$, direct multiplication gives $P^T=P$ and $P^2=P$, and its image is the row space of $G_2$, of dimension $p-1$; [F9] identifies it as an orthogonal projection. For $p=1$, use the empty-row convention $G_2=0$, $P=0$. In either case define the Borel scalar $Q=g(I_m-P)g^T$ on the whole matrix space. On the full-probability event that $G$ has full row rank, the block matrix $W=G{G}^T=\left(\begin{array}{cc}g{g}^T& g{G}_2^T\\ G_2{g}^T& C\end{array}\right)$ has invertible lower block $C$ when $p\ge 2$, and solving $Wx=e_1$ gives $e_1^T{W}^{-1}{e}_1=(g{g}^T-g{G}_2^T{C}^{-1}{G}_2{g}^T{)}^{-1}$. Thus for $p\ge 2$ this $Q$ equals $(e_1^T{W}^{-1}{e}_1{)}^{-1}$ on the full-probability event. For $p=1$, $W=g{g}^T$ and the same identity reads $Q=g{g}^T=(e_1^T{W}^{-1}{e}_1{)}^{-1}$.

The entries of $G$ are independent standard normals, so its first row $g$ is independent of the remaining rows $G_2$ by [F7]. For each fixed full-row-rank value of $G_2$, $I_m-P$ is an orthogonal projection of rank $m-p+1$. Therefore [F4] gives the section law $Q\sim {\chi }_{m-p+1}^2$ for every such $G_2$; the $p=1$ case follows from [F5]. The matrix rank event is Borel because it is determined by continuous minors, and the extended map $P(G_2)$ is Borel by step 2.1. The exceptional set has probability zero by [F2]. Applying the product-law identity [F8] to these Borel sections shows that the unconditional law of the Borel extension $Q=g(I_m-P)g^T$ is also ${\chi }_{m-p+1}^2$.

For any fixed unit vector $u\in {\mathbb{R}}^p$, [F10] supplies an orthogonal matrix $U$ with $Uu=e_1$. By [F3] and [F7], applying $U$ to each independent column preserves their joint standard-normal product law; hence $UW{U}^T$ has the same law as $W$. Let $h(t)=t^{-1}$ for $t>0$ and $h(t)=0$ otherwise. Since $J(W)=W^{-1}$ almost surely by step 1.1 and $W\succ 0$ almost surely, $h(u^{T}J(W)u)$ has the same law as $Q$ in step 3.1 for every unit $u$, using the orthogonal invariance just proved. Set $U_Z=Z\mathrm{/}\mathrm{\parallel }Z\mathrm{\parallel }$ when $Z\mathrm{\ne }0$ and let $U_Z=e_1$ when $Z=0$. The map $z\mapsto U_z$ is Borel, and $Q_Z:=h(U_Z^{T}J(W)U_Z)$ is Borel because $J$ and $h$ are Borel; $Z=0$ is null by [F5]–[F6]. For Borel sets $A,B$, independence and the product law give $\mathbb{P}(\mathrm{\parallel }Z{\mathrm{\parallel }}^2\in A,Q_Z\in B)={\int }_{\{z:\mathrm{\parallel }z{\mathrm{\parallel }}^2\in A\}}{\mathbb{P}}_W(h(u_z^{T}J(W)u_z)\in B)d{\mathbb{P}}_Z(z),$ where $u_z=z\mathrm{/}\mathrm{\parallel }z\mathrm{\parallel }$ for $z\mathrm{\ne }0$ and $u_0=e_1$. The inner probability is the same for every unit $u_z$, so this factors as $\mathbb{P}(\mathrm{\parallel }Z{\mathrm{\parallel }}^2\in A)\mathbb{P}(Q\in B)$. Thus $Q_Z$ is independent of $\mathrm{\parallel }Z{\mathrm{\parallel }}^2$ and has the same ${\chi }_{m-p+1}^2$ law as $Q$ in step 3.1.

By [F5], $\mathrm{\parallel }Z{\mathrm{\parallel }}^2\sim {\chi }_p^2$ and it is independent of $Q_Z\sim {\chi }_{m-p+1}^2$ by step 4.1. Therefore $\frac{m-p+1}{p}{Z}^T{W}^{-1}Z=\frac{\mathrm{\parallel }Z{\mathrm{\parallel }}^2\mathrm{/}p}{Q_Z\mathrm{/}(m-p+1)}$ has law $F_{p,m-p+1}$ by the defining Fisher ratio; its denominator is positive almost surely by [F6].

Define $Y_i={\mathrm{\Sigma }}^{-1\mathrm{/}2}(X_i-\mu )$. The positive root exists by [F10], and if ${\mathrm{\Sigma }}^{1\mathrm{/}2}v=0$ then $v^T\mathrm{\Sigma }v=\mathrm{\parallel }{\mathrm{\Sigma }}^{1\mathrm{/}2}v{\mathrm{\parallel }}^2=0$, so $\mathrm{\Sigma }\succ 0$ makes the root invertible. Whitening is a continuous, hence Borel, map by [F9]; [F6] therefore preserves independence of the sample vectors. By [F2], each $Y_i\sim N_p(0,I_p)$ has independent standard-normal coordinates, and [F11] gives the product joint law across $i$. Thus all $np$ entries $Y_{ia}$ are independent standard normals. Let $u=n^{-1\mathrm{/}2}(1,\dots ,1{)}^T\in {\mathbb{R}}^n$. By [F3], complete $u$ to an orthonormal basis $u_1=u,u_2,\dots ,u_n$ of ${\mathbb{R}}^n$. For each coordinate $a\in \{1,\dots ,p\}$, the sample-coordinate vector $(Y_{1a},\dots ,Y_{na}{)}^T$ is therefore standard normal in ${\mathbb{R}}^n$, and these $p$ vectors are independent.

Let $Y=[Y_1\mathrm{\mid }\cdots \mathrm{\mid }{Y}_n]$ and, for each $j,a$, define $Y_{ja}^{\mathrm{\prime }}=u_j^T(Y_{1a},\dots ,Y_{na}{)}^T$. Applying the orthogonal invariance [F4] separately to the $p$ independent coordinate vectors gives independent standard-normal entries within each transformed coordinate vector; [F6] preserves independence between those $p$ vectors under their separate orthogonal maps. Thus all $np$ entries $Y_{ja}^{\mathrm{\prime }}$ are independent standard normals, and the $p$-vectors $Y_1^{\mathrm{\prime }},\dots ,Y_n^{\mathrm{\prime }}$ formed from their coordinates are independent $N_p(0,I_p)$ vectors.

Since $u_1=u$, $Y_1^{\mathrm{\prime }}=\sqrt{n}{n}^{-1}{\sum }_i{Y}_i=\sqrt{n}{\mathrm{\Sigma }}^{-1\mathrm{/}2}(\bar{X}-\mu )$. Its law and [F2] give $\sqrt{n}(\bar{X}-\mu )\sim N_p(0,\mathrm{\Sigma })$. The vector $Y_1^{\mathrm{\prime }}$ is independent of $(Y_2^{\mathrm{\prime }},\dots ,Y_n^{\mathrm{\prime }})$ by [F5].

The matrix $U=[u_1\mathrm{\mid }\cdots \mathrm{\mid }{u}_n]$ is orthogonal and $u_1=u$. Hence $Y{Y}^T$ splits as $Y{Y}^T=Yu{u}^T{Y}^T+{\sum }_{j=2}^{n}Y{u}_j{u}_j^T{Y}^T$. The first term is $n\bar{Y}{\bar{Y}}^T$, and the remaining terms are $Y_j^{\mathrm{\prime }}{Y}_j^{\mathrm{\prime }T}$, so ${\sum }_i(Y_i-\bar{Y})(Y_i-\bar{Y}{)}^T={\sum }_{j=2}^n{Y}_j^{\mathrm{\prime }}{Y}_j^{\mathrm{\prime }T}$. Since $X_i-\bar{X}={\mathrm{\Sigma }}^{1\mathrm{/}2}(Y_i-\bar{Y})$, it follows that $(n-1)S={\mathrm{\Sigma }}^{1\mathrm{/}2}({\sum }_{j=2}^n{Y}_j^{\mathrm{\prime }}{Y}_j^{\mathrm{\prime }T}){\mathrm{\Sigma }}^{1\mathrm{/}2}$. The independent vectors in [F5] and Wishart definition [F7] give $(n-1)S\sim W_p(\mathrm{\Sigma },n-1)$. The sample mean is a function of $Y_1^{\mathrm{\prime }}$, while $S$ is a function of $Y_2^{\mathrm{\prime }},\dots ,Y_n^{\mathrm{\prime }}$; the maps are Borel by [F9], so [F6] gives $\bar{X}\perp S$.

If $n>p$, then $n-1\ge p$ and the residual vectors $Y_2^{\mathrm{\prime }},\dots ,Y_n^{\mathrm{\prime }}$ are independent standard-normal columns. By [F8], their $p\times (n-1)$ matrix has row rank $p$ almost surely. Its sum of column outer products is therefore positive definite; the congruence in step 4.1 and $n-1>0$ show $S\succ 0$ almost surely.

By [F2] and [F3], one observation has density ${\varphi }_{\mu ,\mathrm{\Sigma }}(x)=(2\pi {)}^{-p\mathrm{/}2}(\det \mathrm{\Sigma }{)}^{-1\mathrm{/}2}\exp [-(x-\mu {)}^T{\mathrm{\Sigma }}^{-1}(x-\mu )\mathrm{/}2]$: the independent-coordinate product density follows from [F4], its continuous Gaussian formula is Borel by [F15], and the invertible affine map $z\mapsto \mu +{\mathrm{\Sigma }}^{1\mathrm{/}2}z$ has determinant $\mathrm{\mid }\det {\mathrm{\Sigma }}^{1\mathrm{/}2}\mathrm{\mid }=(\det \mathrm{\Sigma }{)}^{1\mathrm{/}2}$, so [F5] gives the displayed density. Independence in [F1] then makes the sample density its product over $i=1,\dots ,n$. The finite-dimensional Lebesgue measure is sigma-finite by [F13], so the product law is dominated by it and the displayed function is a likelihood density by [F14].

Under $H_0$, the positive root ${\mathrm{\Sigma }}_0^{1\mathrm{/}2}$ is invertible by [F12], so define $Y_i={\mathrm{\Sigma }}_0^{-1\mathrm{/}2}(X_i-\mu )$. By [F1]–[F2] these are iid $N_p(0,I_p)$. Write $C_n=n^{-1}{\sum }_i{Y}_i{Y}_i^T$ and $A_n={\mathrm{\Sigma }}_0^{-1\mathrm{/}2}M{\mathrm{\Sigma }}_0^{-1\mathrm{/}2}=C_n-\bar{Y}{\bar{Y}}^T$. For a symmetric matrix, let $\mathrm{vech}$ list its upper-triangular entries in a fixed order; put $V_i=\mathrm{vech}(Y_i{Y}_i^T-I_p)$. These vectors have mean zero and diagonal covariance: diagonal coordinates have variance $2$, off-diagonal coordinates variance $1$, and distinct coordinates have covariance zero. Indeed, independence and symmetry give zero for any product with an odd coordinate power; integration by parts against [F3]'s density gives $E{Z}^2=1$ and $E{Z}^4=3$ (using ${\varphi }^{\mathrm{\prime }}(z)=-z\varphi (z)$ and vanishing polynomial-times-Gaussian boundary terms), while independence factors mixed even products. These finite moments make $V_i$ square-integrable, so [F7] yields $\sqrt{n}\mathrm{vech}(C_n-I_p)\Rightarrow N(0,D)$ with those diagonal variances. Also $\sqrt{n}\bar{Y}=O_P(1)$ by [F7]–[F8], whence $\sqrt{n}\bar{Y}{\bar{Y}}^T=n^{-1\mathrm{/}2}(\sqrt{n}\bar{Y})(\sqrt{n}\bar{Y}{)}^T=o_P(1)$ by [F8]. Thus $H_n=\sqrt{n}(A_n-I_p)$ converges in distribution to a symmetric Gaussian matrix $H$ whose entries on and above the diagonal are independent, with variances $2$ on the diagonal and $1$ off it.

Taking logarithms of that positive product density gives, up to the parameter-free constant $-np\log (2\pi )\mathrm{/}2$, ${\mathrm{\ell }}_n(\mu ,\mathrm{\Sigma })=-\frac{n}{2}\log \det \mathrm{\Sigma }-\frac{1}{2}{\sum }_{i=1}^n(X_i-\mu {)}^T{\mathrm{\Sigma }}^{-1}(X_i-\mu )\mathrm{.}$ Expanding $X_i-\mu =(X_i-\bar{X})+(\bar{X}-\mu )$ and using ${\sum }_i(X_i-\bar{X})=0$ shows the quadratic sum equals $n(\bar{X}-\mu {)}^T{\mathrm{\Sigma }}^{-1}(\bar{X}-\mu )+\mathrm{tr}({\mathrm{\Sigma }}^{-1}R)$. Since ${\mathrm{\Sigma }}^{-1}\succ 0$, this is minimized uniquely at $\mu =\bar{X}$ for every fixed $\mathrm{\Sigma }$.

By [F6], $M=R\mathrm{/}n\succ 0$ almost surely. The square root $M^{1\mathrm{/}2}$ exists by [F12] and is invertible because $M\succ 0$. At $\mu =\bar{X}$, compare the covariance log likelihood at any $\mathrm{\Sigma }\succ 0$ with its value at $M$. Let $B=M^{1\mathrm{/}2}{\mathrm{\Sigma }}^{-1}{M}^{1\mathrm{/}2}\succ 0$. The trace and determinant identities give ${\mathrm{\ell }}_n(\bar{X},\mathrm{\Sigma })-{\mathrm{\ell }}_n(\bar{X},M)=-\frac{n}{2}\{\mathrm{tr}B-\log \det B-p\}\mathrm{.}$ By [F11], the eigenvalues ${\lambda }_1,\dots ,{\lambda }_p$ of $B$ are positive and the brace is ${\sum }_j({\lambda }_j-\log {\lambda }_j-1)\ge 0$, since $x-\log x-1\ge 0$ for $x>0$ by differentiation and its unique minimum at $1$. Equality holds precisely when all ${\lambda }_j=1$, hence $B=I$ and $\mathrm{\Sigma }=M$. Thus the unrestricted MLE is $(\bar{X},M)$; under $H_0$ the maximizing mean remains $\bar{X}$ by step 2.1.

Substituting the unrestricted and null maxima into [F10], and using $R=nM$, yields $-2\log {\mathrm{\Lambda }}_n=n\{\mathrm{tr}({\mathrm{\Sigma }}_0^{-1}M)+\log \det {\mathrm{\Sigma }}_0-\log \det M-p\}\mathrm{.}$ Since $\det ({\mathrm{\Sigma }}_0^{-1\mathrm{/}2}M{\mathrm{\Sigma }}_0^{-1\mathrm{/}2})=\det M\mathrm{/}\det {\mathrm{\Sigma }}_0$, this is the asserted finite-sample identity.

Since $H_n=O_P(1)$ by [F8], $E_n=A_n-I_p=H_n\mathrm{/}\sqrt{n}$ has Frobenius norm $\mathrm{\parallel }{E}_n{\mathrm{\parallel }}_F=O_P(n^{-1\mathrm{/}2})$, where $\mathrm{\parallel }B{\mathrm{\parallel }}_F=({\sum }_{i,j}{B}_{ij}^2{)}^{1\mathrm{/}2}$. On an event whose probability tends to one, every eigenvalue $e_j$ of the symmetric $E_n$ has $\mathrm{\mid }{e}_j\mathrm{\mid }\le 1\mathrm{/}2$. Taylor's theorem on $[-1\mathrm{/}2,1\mathrm{/}2]$ gives $e-\log (1+e)=e^2\mathrm{/}2+O(\mathrm{\mid }e{\mathrm{\mid }}^3)$ with a uniform finite remainder constant there. Hence the statistic in step 4.1 equals $n{\sum }_{j=1}^p(e_j-\log (1+e_j))=\frac{1}{2}\mathrm{tr}(H_n^2)+O_P(n\mathrm{\parallel }{E}_n{\mathrm{\parallel }}_F^3)=\frac{1}{2}\mathrm{tr}(H_n^2)+o_P(1),$ because $p$ is fixed and $n\mathrm{\parallel }{E}_n{\mathrm{\parallel }}_F^3=O_P(n^{-1\mathrm{/}2})$.

The map $H\mapsto \frac{1}{2}\mathrm{tr}(H^2)$ is continuous, so [F8] and step 1.2 give convergence to $\frac{1}{2}\mathrm{tr}(H^2)$. For the stated independent entries this is ${\sum }_{i=1}^p(H_{ii}\mathrm{/}\sqrt{2}{)}^2+{\sum }_{1\le i<j\le p}{H}_{ij}^2,$ a sum of $p+p(p-1)\mathrm{/}2=p(p+1)\mathrm{/}2$ independent squared standard normals, which has law ${\chi }_{p(p+1)\mathrm{/}2}^2$ by [F9]. Step 5.1 and Slutsky [F8] prove the asserted limit. When $p=1$, the off-diagonal sum is empty and the same count gives one degree of freedom; $p$ is fixed throughout, and no finite-sample calibration follows from this limit.

The statistic $T^2$ in the Statement is a random variable: by [F9], $\{S\succ 0\}$ is Borel; on this set inversion is continuous, so the displayed quadratic form is Borel, and setting it to zero on the Borel complement preserves measurability. Let $Z=\sqrt{n}{\mathrm{\Sigma }}^{-1\mathrm{/}2}(\bar{X}-{\mu }_0)$. The positive root is invertible by [F7]; under the null, [F1] and [F3] therefore give $Z\sim N_p(0,I_p)$. Moreover, $Z$ and $W:={\mathrm{\Sigma }}^{-1\mathrm{/}2}(n-1)S{\mathrm{\Sigma }}^{-1\mathrm{/}2}$ are independent: $\bar{X}$ and $S$ are independent by [F1], and the separate whitening maps are continuous, hence Borel, so [F8] preserves independence. The congruence in [F1] and [F2] gives $W\sim W_p(I_p,n-1)$; it is positive definite almost surely by [F1].

On the event $S\succ 0$ (which has probability one by [F1]), substitute $S={\mathrm{\Sigma }}^{1\mathrm{/}2}W{\mathrm{\Sigma }}^{1\mathrm{/}2}\mathrm{/}(n-1)$ and $\bar{X}-{\mu }_0={\mathrm{\Sigma }}^{1\mathrm{/}2}Z\mathrm{/}\sqrt{n}$ into the definition. Cancelling the inverse congruence gives $T^2=(n-1)Z^T{W}^{-1}Z\mathrm{.}$

Apply [F4] with $m=n-1\ge p$. It gives $\frac{n-p}{p}{Z}^T{W}^{-1}Z\sim F_{p,n-p}$. Multiplying the identity in step 2.1 by $(n-p)\mathrm{/}(p(n-1))$ proves the asserted law. Since $S\succ 0$ almost surely, the off-domain zero convention affects only a null event.

Put $m=n-1\ge 1$. By [F1]–[F3], the law of $(n-1)S$ is the law of ${\sum }_{j=1}^m{X}_j{X}_j^T$ for independent $X_j={\mathrm{\Sigma }}_0^{1\mathrm{/}2}{Z}_j$ with all $pm$ scalar coordinates independent standard normals.

For each summand, cyclicity of the finite trace and ${\mathrm{\Sigma }}_0^{-1\mathrm{/}2}{\mathrm{\Sigma }}_0^{1\mathrm{/}2}=I_p$ give $\mathrm{tr}({\mathrm{\Sigma }}_0^{-1}{X}_j{X}_j^T)=X_j^T{\mathrm{\Sigma }}_0^{-1}{X}_j=Z_j^T{Z}_j\mathrm{.}$ By linearity of trace, the target statistic therefore has the law of ${\sum }_{j=1}^m{\sum }_{a=1}^p{Z}_{ja}^2$.

The $pm=p(n-1)$ coordinates in step 2.1 are independent standard normals, so their squared sum has law ${\chi }_{pm}^2$ by [F4]. This argument does not require $m\ge p$: if $m<p$, the matrix scatter may be singular, but its trace and the sum of squares remain defined; $m=1$ and $p=1$ are both included. The statistic is distinct from the likelihood-ratio statistic, whose formula contains a log determinant when the scatter is nonsingular.