The Spectral Theorem, Positive Operators and Singular Value Decomposition

By the definition of self-adjointness in Self-adjoint and normal endomorphisms of a finite-dimensional real or complex inner product space, $T$ is self-adjoint exactly when $T=T^{\ast }$. By [L1], this is equivalent to $A=[T{]}_{\mathcal{E}}^{\mathcal{E}}=[T^{\ast }{]}_{\mathcal{E}}^{\mathcal{E}}=A^{\ast }$.

By the definition of normality in Self-adjoint and normal endomorphisms of a finite-dimensional real or complex inner product space, $T$ is normal exactly when $T{T}^{\ast }=T^{\ast }T$; using [L2] and then [L1], this is equivalent to $[T{T}^{\ast }{]}_{\mathcal{E}}^{\mathcal{E}}=A[T^{\ast }{]}_{\mathcal{E}}^{\mathcal{E}}=A{A}^{\ast }=[T^{\ast }{]}_{\mathcal{E}}^{\mathcal{E}}A=A^{\ast }A=[T^{\ast }T{]}_{\mathcal{E}}^{\mathcal{E}}$, hence to $A{A}^{\ast }=A^{\ast }A$.

By [L1], there is a basis $(v_1,\dots ,v_n)$ of $V$ in which $[T]$ is upper triangular; equivalently, the flag $0\subset \mathrm{span}(v_1)\subset \mathrm{span}(v_1,v_2)\subset \cdots \subset \mathrm{span}(v_1,\dots ,v_n)=V$ is $T$-invariant by [L2].

Apply [L3] to $(v_1,\dots ,v_n)$ and obtain an orthonormal basis $(e_1,\dots ,e_n)$ such that $\mathrm{span}(e_1,\dots ,e_k)=\mathrm{span}(v_1,\dots ,v_k)$ for every $k$, so the same successive spans are still $T$-invariant.

Because the orthonormal basis $(e_1,\dots ,e_n)$ is adapted to a complete $T$-invariant flag, [L2] shows that the matrix of $T$ in that basis is upper triangular.

Because $A$ is upper triangular, the $n$-th diagonal entry of $A{A}^{\ast }$ is ${\sum }_{j\ge n}\mathrm{\mid }{a}_{nj}{\mathrm{\mid }}^2=\mathrm{\mid }{a}_{nn}{\mathrm{\mid }}^2$, while the $n$-th diagonal entry of $A^{\ast }A$ is ${\sum }_{i\le n}\mathrm{\mid }{a}_{in}{\mathrm{\mid }}^2$; [L1] makes these equal, so $a_{in}=0$ for every $i<n$.

More generally, if every entry to the right of $a_{kk}$ in row $k$ is already zero, then the $k$-th diagonal entries of $A{A}^{\ast }$ and $A^{\ast }A$ are $\mathrm{\mid }{a}_{kk}{\mathrm{\mid }}^2$ and ${\sum }_{i\le k}\mathrm{\mid }{a}_{ik}{\mathrm{\mid }}^2$, so [L1] forces $a_{ik}=0$ for every $i<k$.

Starting from $k=n$ in step 1.1 and descending through $k=n-1,\dots ,1$, step 2.1 shows that every entry above the diagonal is zero. Thus $A$ is diagonal.

Assume $T$ is normal. By [L1], the characteristic polynomial of $T$ splits over $\mathbb{C}$, so [L2] gives an orthonormal basis in which the matrix of $T$ is upper triangular. Because $T$ is normal, [L4] makes that matrix normal, and then [L3] makes it diagonal. Hence the chosen orthonormal basis consists of eigenvectors of $T$.

Conversely, assume $(e_1,\dots ,e_n)$ is an orthonormal basis of eigenvectors of $T$, with $T{e}_j={\lambda }_j{e}_j$. Then the matrix of $T$ in that basis is the diagonal matrix $\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$, and every diagonal matrix commutes with its conjugate transpose. By [L4], $T$ is normal.

Choose an orthonormal basis of $V$ and let $A\in M_n(\mathbb{R})$ be the matrix of $T$. By [L1], $A{A}^{\mathsf{T}}=A^{\mathsf{T}}A$. Because $A$ has real entries, its conjugate transpose over $\mathbb{C}$ is still $A^{\mathsf{T}}$, so $A$ is a normal complex matrix on ${\mathbb{C}}^n$ with the standard Hermitian inner product from [L3]. Therefore [L2] gives an orthonormal eigenbasis of ${\mathbb{C}}^n$ for $A$.

If $\lambda \in \mathbb{R}$ is an eigenvalue and $v=u+iw$ is a corresponding complex eigenvector, then $Au+iAw=Av=\lambda v=\lambda u+i\lambda w$, so $Au=\lambda u$ and $Aw=\lambda w$; hence one of $u,w$ is a nonzero real eigenvector, and repeating inside each real eigenspace yields orthonormal real eigenvectors for the real eigenvalues.

If $\lambda =a+ib$ with $b\mathrm{\ne }0$ and $v=u+iw$ is a unit eigenvector, then $A\bar{v}=\bar{\lambda }\bar{v}$ because $A$ is real; since $\lambda \mathrm{\ne }\bar{\lambda }$, the vectors $v$ and $\bar{v}$ are orthogonal in the orthonormal eigenbasis from step 1.1, so writing out $⟨v,\bar{v}⟩=0$ and using [L4] gives $\mathrm{\parallel }u{\mathrm{\parallel }}^2-\mathrm{\parallel }w{\mathrm{\parallel }}^2+2i⟨u,w⟩=0$, hence $\mathrm{\parallel }u\mathrm{\parallel }=\mathrm{\parallel }w\mathrm{\parallel }$ and $⟨u,w⟩=0$. Also $A(u+iw)=(a+ib)(u+iw)=(au-bw)+i(bu+aw)$, so $Au=au-bw$ and $Aw=bu+aw$; after normalising $e_1=u\mathrm{/}\mathrm{\parallel }u\mathrm{\parallel }$ and $e_2=w\mathrm{/}\mathrm{\parallel }w\mathrm{\parallel }$, the matrix of $A$ on $\mathrm{span}(e_1,e_2)$ is $\left(\begin{array}{cc}a& b\\ -b& a\end{array}\right)$.

The orthonormal complex eigenbasis from step 1.1 splits into real eigenvectors and conjugate pairs. Step 1.2 replaces each real eigenvector by a real one, and step 2.1 replaces each conjugate pair by an orthonormal real pair spanning the same real invariant plane. Collecting these mutually orthogonal pieces yields an orthonormal real basis with the stated block-diagonal matrix.

By [L1] and [L2], there is an orthonormal basis in which the matrix of $T$ is block diagonal with $1\times 1$ real blocks and $2\times 2$ blocks $B=\left(\begin{array}{cc}a& b\\ -b& a\end{array}\right)$.

Because $T$ is self-adjoint, [L3] says that the same matrix is symmetric. A block $B$ above is symmetric exactly when $b=0$, so every $2\times 2$ block collapses to the scalar block $[a]$. Therefore the whole matrix is diagonal.

A diagonal matrix acts on each basis vector by scalar multiplication, so the orthonormal basis from step 2.1 is an orthonormal eigenbasis of $T$.

By [L1], $V$ has an orthonormal eigenbasis. If $V=0$, then there are no eigenvalues and $T=0$ is the empty sum. Otherwise $V=E_{{\lambda }_1}(T)\oplus \cdots \oplus E_{{\lambda }_r}(T)$ with the summands pairwise orthogonal; if $P_j$ is the orthogonal projection onto $E_{{\lambda }_j}(T)$ and $v=v_1+\cdots +v_r$ with $v_j\in E_{{\lambda }_j}(T)$, then $Tv={\lambda }_1{v}_1+\cdots +{\lambda }_r{v}_r={\sum }_{j=1}^r{\lambda }_j{P}_{j}v$, hence $T={\sum }_{j=1}^r{\lambda }_j{P}_j$.

Because the eigenspaces are pairwise orthogonal and the $P_j$ are the corresponding orthogonal projections, one has $P_j^2=P_j$, $P_j^{\ast }=P_j$, and $P_i{P}_j=0$ for $i\mathrm{\ne }j$, so the $P_j$ are pairwise orthogonal projections in the sense of [L2].

Since $T$ is diagonalisable with distinct eigenvalues, its primary decomposition is exactly the direct sum of the eigenspaces $E_{{\lambda }_j}(T)$. Therefore [L3] identifies each projection onto $E_{{\lambda }_j}(T)$ with a polynomial in $T$.

By definition $f(T)={\sum }_{j}f({\lambda }_j)P_j$ and $g(T)={\sum }_{j}g({\lambda }_j)P_j$, so $(f+g)(T)={\sum }_j(f({\lambda }_j)+g({\lambda }_j))P_j=f(T)+g(T)$; using $P_i{P}_j=0$ for $i\mathrm{\ne }j$ and $P_j^2=P_j$ from [L1], one also gets $f(T)g(T)={\sum }_{j}f({\lambda }_j)g({\lambda }_j)P_j=(fg)(T)$.

Because each $P_j$ is self-adjoint by [L1], one has $f(T{)}^{\ast }={({\sum }_{j}f({\lambda }_j)P_j)}^{\ast }={\sum }_j\overline{f({\lambda }_j)}{P}_j=\bar{f}(T)$.

Put ${\mu }_1,\dots ,{\mu }_s$ for the distinct values taken by $f$ on $\sigma (T)$ and define $Q_{\mathrm{\ell }}:={\sum }_{f({\lambda }_j)={\mu }_{\mathrm{\ell }}}{P}_j$; then the $Q_{\mathrm{\ell }}$ are pairwise orthogonal projections, $f(T)={\sum }_{\mathrm{\ell }=1}^s{\mu }_{\mathrm{\ell }}{Q}_{\mathrm{\ell }}$, and applying the same definition of functional calculus once more gives $h(f(T))={\sum }_{\mathrm{\ell }=1}^{s}h({\mu }_{\mathrm{\ell }})Q_{\mathrm{\ell }}={\sum }_{j}h(f({\lambda }_j))P_j=(h\circ f)(T)$.

Choose an algebraic closure $\mathrm{\Omega }\mathrm{/}F$ as in An algebraic closure of a field. Let $q$ be the product of the distinct monic irreducible factors of the characteristic polynomial of $T$, and let $E\subseteq \mathrm{\Omega }$ be its splitting field. By [L1], the polynomial $q$ is separable, so [L4] makes $E\mathrm{/}F$ finite Galois with fixed field $E^{\mathrm{Gal}(E\mathrm{/}F)}=F$. The characteristic polynomial splits over $E$, and [L2] gives Jordan form for the matrix of $T$ over $E$. Thus $E^n$ decomposes as the direct sum of generalized eigenspaces $G_{\lambda }$, and $(T_E-\lambda I){\mathrm{\mid }}_{G_{\lambda }}$ is nilpotent.

Let $P_{\lambda }$ be the projection onto $G_{\lambda }$ along the sum of the other generalized eigenspaces. By [L3], each $P_{\lambda }$ is a polynomial in $T_E$. Define $S_E:={\sum }_{\lambda }\lambda P_{\lambda }$ and $N_E:=T_E-S_E={\sum }_{\lambda }(T_E-\lambda I)P_{\lambda }$. Then $T_E=S_E+N_E$, the operators commute because they are polynomials in $T_E$, the minimal polynomial of $S_E$ divides ${\prod }_{\lambda }(x-\lambda )$ and therefore has distinct roots, so [L5] makes $S_E$ semisimple, and $N_E$ is nilpotent because its restriction to each $G_{\lambda }$ is $(T_E-\lambda I){\mathrm{\mid }}_{G_{\lambda }}$.

Suppose also that $T_E=S_E^{\mathrm{\prime }}+N_E^{\mathrm{\prime }}$ with $S_E^{\mathrm{\prime }}$ semisimple, $N_E^{\mathrm{\prime }}$ nilpotent, and $S_E^{\mathrm{\prime }}{N}_E^{\mathrm{\prime }}=N_E^{\mathrm{\prime }}{S}_E^{\mathrm{\prime }}$. Because $N_E^{\mathrm{\prime }}$ commutes with $T_E$, every generalized eigenspace $G_{\lambda }$ from step 1.1 is invariant under both $S_E^{\mathrm{\prime }}$ and $N_E^{\mathrm{\prime }}$. On $G_{\lambda }$, the operator $T_E$ has only the eigenvalue $\lambda$, while $N_E^{\mathrm{\prime }}$ has only the eigenvalue $0$; hence the semisimple operator $S_E^{\mathrm{\prime }}{\mathrm{\mid }}_{G_{\lambda }}$ has only the eigenvalue $\lambda$, so $S_E^{\mathrm{\prime }}=\lambda I$ on $G_{\lambda }$ and therefore $N_E^{\mathrm{\prime }}=T_E-\lambda I$ there. Thus $S_E^{\mathrm{\prime }}=S_E$ and $N_E^{\mathrm{\prime }}=N_E$.

Let $\sigma \in \mathrm{Gal}(E\mathrm{/}F)$. Acting entrywise, $\sigma$ fixes the matrix of $T_E$ and permutes the roots of its characteristic polynomial. Applying $\sigma$ to the construction in step 2.1 therefore produces another commuting semisimple-plus-nilpotent decomposition of $T_E$. Uniqueness in step 3.1 forces $\sigma (S_E)=S_E$ and $\sigma (N_E)=N_E$.

Choose an $F$-basis $B_1,\dots ,B_d$ of the finite-dimensional algebra $F[T]$. The same matrices form an $E$-basis of $E[T_E]$. Write $S_E={\sum }_i{c}_i{B}_i$ with $c_i\in E$. Step 4.1 and uniqueness of coordinates give $\sigma (c_i)=c_i$ for every $\sigma \in \mathrm{Gal}(E\mathrm{/}F)$, so [L4] gives $c_i\in F$. Hence $S_E$ is the scalar extension of a polynomial $T_s=p(T)$ with $p\in F[x]$. The same argument applies to $N_E$, giving $T_n=q(T)$ with $q\in F[x]$. Extending the identities of step 2.1 back down to $F$ gives $T=T_s+T_n$ and $T_s{T}_n=T_n{T}_s$; semisimplicity and nilpotence descend because their minimal-polynomial identities have coefficients in $F$. Uniqueness follows after extension to $E$ from step 3.1.

For a linear map $T$, the form $B_T(u,v)=⟨Tu,v⟩$ is bilinear. Conversely, let $B$ be a bilinear form on $V$. By [L1], the assignment $u\mapsto B(u,-)$ is a linear map $V\to V^{\ast }$. For each $u\in V$, [L2] gives a unique vector $Tu\in V$ such that $B(u,v)=⟨Tu,v⟩$ for every $v\in V$. If $a,b\in \mathbb{R}$ and $u,u^{\mathrm{\prime }}\in V$, then for every $v$ one has $⟨T(au+b{u}^{\mathrm{\prime }}),v⟩=B(au+b{u}^{\mathrm{\prime }},v)=aB(u,v)+bB(u^{\mathrm{\prime }},v)=⟨aTu+bT{u}^{\mathrm{\prime }},v⟩,$ so uniqueness in [L2] gives $T(au+b{u}^{\mathrm{\prime }})=aTu+bT{u}^{\mathrm{\prime }}$. Thus $T$ is linear, and the correspondence $T\mapsto B_T$ is bijective on all linear maps and bilinear forms. In the real case, $T$ is self-adjoint exactly when $⟨Tu,v⟩=⟨u,Tv⟩$ for all $u,v$, and symmetry of the inner product makes this exactly the condition $B_T(u,v)=B_T(v,u)$. Thus self-adjoint endomorphisms correspond exactly to symmetric bilinear forms.

For every $v\in V$, one has $B_T(v,v)=⟨Tv,v⟩$. Therefore the weak inequality in Non-negative and positive operators is exactly the positive-semidefinite condition in [L3], and the strict inequality on nonzero vectors is exactly the positive-definite condition in [L3].

Assume $T$ is non-negative. Then $T$ is self-adjoint by Non-negative and positive operators. If $V$ is real, [L1] gives an orthonormal eigenbasis; if $V$ is complex, the identity $T=T^{\ast }$ yields $T{T}^{\ast }=T^{\ast }T$, so [L2] gives an orthonormal eigenbasis. For any eigenvector $v\mathrm{\ne }0$ with $Tv=\lambda v$, the equality $⟨Tv,v⟩=\lambda \parallel v{\parallel }^2$ and non-negativity give $\lambda \ge 0$, and self-adjointness makes $\lambda$ real. Thus claim 1 implies claim 2.

Assume claim 3. If the diagonal entries are ${\lambda }_1,\dots ,{\lambda }_n\ge 0$, then [L3] gives square roots $\sqrt{{\lambda }_j}\ge 0$; the diagonal operator $S$ with diagonal entries $\sqrt{{\lambda }_j}$ satisfies $S^{\ast }S=T$. Thus claim 3 implies claim 4.

In the orthonormal eigenbasis from step 1.1, the matrix of $T$ is diagonal with those nonnegative real eigenvalues on the diagonal. Thus claim 2 implies claim 3.

Assume claim 4, so $T=S^{\ast }S$. Then for every $v\in V$ one has $⟨Tv,v⟩=⟨S^{\ast }Sv,v⟩=⟨Sv,Sv⟩\ge 0$, and for all $u,v$ one has $⟨Tu,v⟩=⟨Su,Sv⟩=⟨u,Tv⟩$. Hence $T$ is self-adjoint and non-negative. Thus claim 4 implies claim 1.

By [L1], there is an orthonormal basis $(e_1,\dots ,e_n)$ of $V$ and numbers ${\lambda }_1,\dots ,{\lambda }_n\ge 0$ such that $T{e}_j={\lambda }_j{e}_j$. By [L2], each ${\lambda }_j$ has a nonnegative square root. Define $R$ by $R{e}_j=\sqrt{{\lambda }_j}{e}_j$. Then $R^2{e}_j={\lambda }_j{e}_j=T{e}_j$ for every $j$, so $R^2=T$; the same basis shows that $R$ is self-adjoint with nonnegative eigenvalues, hence non-negative by [L1].

Let $Q$ be another non-negative square root of $T$. Because $Q^2=T$, each eigenspace $E_{\lambda }(T)$ is $Q$-invariant. On $E_{\lambda }(T)$ one has $Q^2=\lambda I$, and [L1] applied to the non-negative operator $Q{\mathrm{\mid }}_{E_{\lambda }(T)}$ shows that all its eigenvalues are nonnegative; by [L2], the only nonnegative number whose square is $\lambda$ is $\sqrt{\lambda }$, so $Q=\sqrt{\lambda }I$ on $E_{\lambda }(T)$. Thus $Q$ acts on every basis vector $e_j$ exactly as $R$ does, and therefore $Q=R$.

By [L1], $V$ has an orthonormal eigenbasis consisting of eigenvectors of $T$, and every eigenvalue ${\lambda }_j$ is a nonnegative real number. Interpolation on the finite set $\{{\lambda }_1,\dots ,{\lambda }_r\}$ gives a polynomial $p\in \mathbb{R}[x]$ with $p({\lambda }_j)=\sqrt{{\lambda }_j}$ for every $j$.

On each eigenspace $E_{{\lambda }_j}(T)$, the operator $p(T)$ acts as multiplication by $p({\lambda }_j)=\sqrt{{\lambda }_j}$. Therefore $p(T{)}^2$ acts as multiplication by ${\lambda }_j$, so $p(T{)}^2=T$; the same eigenbasis shows that $p(T)$ is non-negative. By [L2], $p(T)$ is the unique non-negative square root of $T$.

The operator $T^{\ast }T$ is determined by $T$, and [L2] gives it a unique non-negative square root. Therefore the operator $\mathrm{\mid }T\mathrm{\mid }=\sqrt{T^{\ast }T}$ in [L1] is determined uniquely by $T$.

The singular values are the eigenvalues of that uniquely determined operator $\mathrm{\mid }T\mathrm{\mid }$, counted with multiplicity. Hence they depend only on $T$.

By [L1] and [L2], the non-negative operator $\mathrm{\mid }T\mathrm{\mid }$ on $V$ has an orthonormal eigenbasis $(e_1,\dots ,e_n)$ with $\mathrm{\mid }T\mathrm{\mid }{e}_j=s_j{e}_j$ and $s_1\ge \cdots \ge s_r>0=s_{r+1}=\cdots =s_n$. Because $\mathrm{\mid }T{\mathrm{\mid }}^2=T^{\ast }T$, one has $⟨T^{\ast }T{e}_j,e_j⟩=s_j^2$ and therefore $\parallel T{e}_j\parallel =s_j$.

For each $j\le r$, define $f_j:=s_j^{-1}T{e}_j$. Then $\parallel f_j\parallel =1$, and for $i\mathrm{\ne }j$ one has $⟨f_i,f_j⟩=(s_i{s}_j{)}^{-1}⟨T^{\ast }T{e}_i,e_j⟩=(s_i{s}_j{)}^{-1}{s}_i^2⟨e_i,e_j⟩=0$. For $j>r$, step 1.1 gives $\parallel T{e}_j\parallel =s_j=0$, so $T{e}_j=0$.

The orthonormal set $(f_1,\dots ,f_r)$, which may be empty when $r=0$, extends to an orthonormal basis $(f_1,\dots ,f_m)$ of $W$ by [L3]. If $v={\sum }_{j=1}^n⟨v,e_j⟩e_j$, then step 2.1 gives $Tv={\sum }_{j=1}^n⟨v,e_j⟩T{e}_j={\sum }_{j=1}^r{s}_j⟨v,e_j⟩f_j$, and when $r=0$ this is the empty sum $0$.

By [L1], one has $Tv=0$ exactly when $⟨v,e_j⟩=0$ for $1\le j\le r$, so $\ker T=\mathrm{span}(e_{r+1},\dots ,e_n)$ and therefore $\mathrm{nullity}T=n-r$.

By [L2], $\mathrm{rank}T+\mathrm{nullity}T=n$. Combining this with step 1.1 gives $\mathrm{rank}T=r$, the number of positive singular values.