Inner Product Spaces, Gram-Schmidt, Projections and Adjoints

Distributivity of the finite sums in [L1] gives linearity in the first variable. In the complex case [L2] gives conjugate-linearity in the second and conjugate symmetry; in the real case conjugation is the identity.

On the diagonal, the real formula is $\sum x_k^2$ and the complex formula is $\sum \mathrm{\mid }{z}_k{\mathrm{\mid }}^2$. Each is nonnegative and vanishes only when every coordinate is zero, which by [L3] means the vector is zero.

Hence all the axioms in [L4] hold. When $n=0$, the sum is empty and equals $0$, while the zero vector is the only vector, so definiteness is valid.

If $⟨u,v⟩=0$ for every $v$, take $v=u$; [L1] gives $u=0$.

Sesquilinearity and [L3] give $⟨\lambda v,\lambda v⟩=\lambda \bar{\lambda }⟨v,v⟩=\mathrm{\mid }\lambda {\mathrm{\mid }}^2\parallel v{\parallel }^2$. Both $\parallel \lambda v\parallel$ and $\mathrm{\mid }\lambda \mathrm{\mid }\parallel v\parallel$ are nonnegative, so uniqueness in [L2] gives their equality.

If $v=0$, both sides are zero and the pair is dependent. Suppose $v\mathrm{\ne }0$, put $c=⟨u,v⟩\mathrm{/}⟨v,v⟩$, and use [L1] to expand $0\le ⟨u-cv,u-cv⟩=\parallel u{\parallel }^2-\mathrm{\mid }⟨u,v⟩{\mathrm{\mid }}^2\mathrm{/}\parallel v{\parallel }^2$.

Conversely, if $u,v$ are dependent and neither is zero, write $u=cv$; then $\mathrm{\mid }⟨u,v⟩\mathrm{\mid }=\mathrm{\mid }c\mathrm{\mid }\parallel v{\parallel }^2=\parallel u\parallel \parallel v\parallel$. If either is zero, equality is immediate.

Multiplying step 1.1 by the positive number $\parallel v{\parallel }^2$ gives $\mathrm{\mid }⟨u,v⟩{\mathrm{\mid }}^2\le \parallel u{\parallel }^2\parallel v{\parallel }^2$. Since both sides of the desired inequality are nonnegative, factoring the difference of their squares gives the stated inequality.

Under $v\mathrm{\ne }0$, equality in step 2.1 holds exactly when $⟨u-cv,u-cv⟩=0$, which by [L1] is exactly $u=cv$. Thus equality implies dependence. The already separated case $v=0$ does too.

Steps 2.1, 3.1, and 1.2 prove the inequality and both equality directions.

Nonnegativity and definiteness follow directly from [L1], and homogeneity is [L2].

Expanding and using conjugate symmetry gives $\parallel u+v{\parallel }^2=\parallel u{\parallel }^2+2\mathrm{Re}⟨u,v⟩+\parallel v{\parallel }^2$. From [L4], $\mathrm{\mid }z{\mathrm{\mid }}^2=(\mathrm{Re}z{)}^2+(\mathrm{Im}z{)}^2$, so $\mathrm{Re}z\le \mathrm{\mid }z\mathrm{\mid }$; now [L3] makes the expansion at most $(\parallel u\parallel +\parallel v\parallel {)}^2$.

Both quantities in step 1.2 are nonnegative. If the left were larger, their squared order would also be larger, a contradiction. Hence the triangle inequality holds.

Expanding by [L1] and [L2] gives $\parallel u+v{\parallel }^2=\parallel u{\parallel }^2+⟨u,v⟩+\overline{⟨u,v⟩}+\parallel v{\parallel }^2$. Orthogonality removes the middle terms and proves Pythagoras.

Expanding $\parallel u-v{\parallel }^2$ changes the signs of both middle terms. Adding this expansion to step 1.1 proves the parallelogram identity; subtracting gives $4\mathrm{Re}⟨u,v⟩$.

Over $\mathbb{R}$, the real part is the scalar itself, giving claim 3. Over $\mathbb{C}$, the same expansion with $iv$ gives $\parallel u+iv{\parallel }^2-\parallel u-iv{\parallel }^2=4\mathrm{Im}⟨u,v⟩$ under the linear-first convention. Combining real and imaginary parts gives claim 4.

Substituting [L1] into [L2] gives the displayed finite-sum inequality term for term.

Coordinate vectors are linearly dependent exactly when there is a nonzero scalar pair $(\lambda ,\mu )$ with $\lambda a_k=\mu b_k$ for all $k$, so the equality condition agrees with [L3]. For $n=0$, both sides are zero and the empty lists satisfy the symmetric proportionality condition.

If $r=0$, the independence condition is vacuous. Suppose $r>0$ and ${\sum }_{i<r}{a}_i{v}_i=0$.

For each $j<r$, pair the equality in step 1.1 with $v_j$. Linearity and [L1] give $0={\sum }_{i<r}{a}_i⟨v_i,v_j⟩=a_j⟨v_j,v_j⟩$.

By [L2], $⟨v_j,v_j⟩\mathrm{\ne }0$, so $a_j=0$. This holds for every $j$, and [L3] proves independence.

For $r=0$ there is nothing to construct, and the successive-span assertion at $k=0$ is equality of zero subspaces.

Suppose $e_0,\dots ,e_{k-1}$ have been constructed orthonormally with the required span equalities. Define $u_k$ by the displayed formula. For $i<k$, linearity and orthonormality give $⟨u_k,e_i⟩=⟨v_k,e_i⟩-⟨v_k,e_i⟩=0$.

If $u_k=0$, then $v_k$ lies in $\mathrm{span}(e_0,\dots ,e_{k-1})=\mathrm{span}(v_0,\dots ,v_{k-1})$, say $v_k={\sum }_{i<k}{\mu }_i{v}_i$; then the scalars ${\lambda }_i={\mu }_i$ for $i<k$, ${\lambda }_k=-1_F$ and ${\lambda }_i=0_F$ for $i>k$ satisfy ${\sum }_i{\lambda }_i{v}_i=0_V$ with ${\lambda }_k\mathrm{\ne }{0}_F$, contradicting the independence of $v$ through [L2]. Hence $u_k\mathrm{\ne }0$, and [L3] makes $e_k=u_k\mathrm{/}\parallel u_k\parallel$ a unit vector orthogonal to its predecessors.

The formula for $u_k$ shows $e_k$ lies in $\mathrm{span}(v_0,\dots ,v_k)$, while its rearrangement shows $v_k$ lies in $\mathrm{span}(e_0,\dots ,e_k)$. Together with the induction hypothesis these give both inclusions in the span equality at $k+1$.

Induction constructs the stated list and proves every successive-span equality. Its vectors are nonzero and orthogonal, so [L4] also confirms their independence; their unit norms make the list orthonormal.

Choose a finite basis $(v_0,\dots ,v_{r-1})$ of $V$ using [L1].

Apply [L2]. The resulting orthonormal list has the same span as the basis, namely $V$, and therefore is an orthonormal basis. This also covers $r=0$.

Put $p={\sum }_{i<r}⟨v,e_i⟩e_i$. For each $j<r$, [L1] gives $⟨v-p,e_j⟩=0$, so $v-p$ is orthogonal to $p$.

By [L2], $\parallel v{\parallel }^2=\parallel p{\parallel }^2+\parallel v-p{\parallel }^2$. A second use of orthonormality gives $\parallel p{\parallel }^2={\sum }_{i<r}\mathrm{\mid }⟨v,e_i⟩{\mathrm{\mid }}^2$, proving Bessel's inequality.

Equality holds in step 2.1 exactly when $\parallel v-p\parallel =0$, hence exactly when $v=p$. By [L3], this is exactly $v$ belonging to the listed span.

If the list is a basis, its span is $V$, so step 3.1 gives the coordinate expansion and the squared-length identity. Substitute the coordinate expansion of $v$ into $⟨v,w⟩$ and use conjugate symmetry to obtain the displayed inner-product formula.

When $r=0$, [L4] makes every displayed sum zero; the list can be a basis only of the zero space, so all assertions remain valid.

By [L1], choose a basis $(w_0,\dots ,w_{s-1})$ of $W$ and extend it to a basis $(w_0,\dots ,w_{s-1},v_s,\dots ,v_{n-1})$ of $V$. Empty initial or terminal blocks cover $W=0$ and $W=V$.

Apply [L2] to this basis, obtaining an orthonormal basis $(e_0,\dots ,e_{n-1})$ with $W=\mathrm{span}(e_0,\dots ,e_{s-1})$. Put $U=\mathrm{span}(e_s,\dots ,e_{n-1})$. Orthonormality and [L3] give $U\subseteq W^{\perp }$.

The orthonormal basis splits every vector as a sum of a vector in $W$ and a vector in $U$, so $V=W+U\subseteq W+W^{\perp }$. The reverse inclusion is automatic.

If $x\in W\cap W^{\perp }$, then [L3] gives $⟨x,x⟩=0$, and positive definiteness gives $x=0$. With step 3.1 this is exactly the pair of conditions in [L4], so $V=W\oplus W^{\perp }$. The decomposition of each $x$ is unique: if $w+u=w^{\mathrm{\prime }}+u^{\mathrm{\prime }}$ with $w,w^{\mathrm{\prime }}\in W$ and $u,u^{\mathrm{\prime }}\in W^{\perp }$, then $w-w^{\mathrm{\prime }}=u^{\mathrm{\prime }}-u$ lies in $W\cap W^{\perp }=\{0_V\}$, so $w=w^{\mathrm{\prime }}$ and $u=u^{\mathrm{\prime }}$.

Apply [L2] to [L1] to obtain $\dim V=\dim W+\dim W^{\perp }$.

Conjugate symmetry shows $W\subseteq W^{\perp \perp }$. Apply step 1.1 first to $W$ and then to $W^{\perp }$ to get $\dim W^{\perp \perp }=\dim V-\dim W^{\perp }=\dim W$.

The inclusion and equal dimensions in step 2.1 imply $W^{\perp \perp }=W$ by [L3]. The same reasoning covers both endpoint subspaces.

Put $p={\sum }_{i<r}⟨v,e_i⟩e_i$. Then $p\in W$, and for every basis vector $e_j$, [L3] and orthonormality give $⟨v-p,e_j⟩=0$. By [L2], this makes $v-p$ orthogonal to all of $W$.

The defining decomposition shows $P_{W}w=w$ for $w\in W$ and $P_{W}z=0$ for $z\in W^{\perp }$. Hence $\mathrm{im}{P}_W=W$, $\ker P_W=W^{\perp }$, and $P_W^2=P_W$.

The uniqueness clause in [L1] gives $P_{W}v=p$, proving the formula. Linearity follows immediately from [L3] and the formula.

For the decomposition $v=P_{W}v+(v-P_{W}v)$, the second summand lies in $W^{\perp }$. Its projection onto $W^{\perp }$ is itself, so $P_{W^{\perp }}v=v-P_{W}v$.

Decompose $v-w=(v-P_{W}v)+(P_{W}v-w)$. The first term lies in $W^{\perp }$ and the second in $W$, so they are orthogonal by [L1].

By [L2], $\parallel v-w{\parallel }^2=\parallel v-P_{W}v{\parallel }^2+\parallel P_{W}v-w{\parallel }^2\ge \parallel v-P_{W}v{\parallel }^2$. Nonnegativity gives the asserted inequality.

Equality holds exactly when $\parallel P_{W}v-w{\parallel }^2=0$, which by positive definiteness is exactly $w=P_{W}v$.

If the list is dependent, choose nonzero coefficients $c_j$ with ${\sum }_{j<r}{c}_j{v}_j=0$ by [L3]. Then $(G\bar{c}{)}_i={\sum }_j⟨v_i,v_j⟩\overline{c_j}=⟨v_i,{\sum }_j{c}_j{v}_j⟩=0$. Since $\bar{c}\mathrm{\ne }0$, $G$ is singular and [L6] gives $\det G=0$.

Suppose the list is independent and $r\ge 1$. Apply [L2], and write each $v_j={\sum }_{i\le j}{R}_{ij}{e}_i$. The resulting $r\times r$ upper-triangular matrix $R$ has diagonal entries $R_{jj}=\parallel u_j\parallel >0$.

Orthonormality and the linear-first convention give $G=R^{\mathsf{T}}\bar{R}$. Because conjugation is a field automorphism by [L5], conjugating $R$ entrywise conjugates its determinant, so $\det \bar{R}=\overline{\det R}$. Since $r\ge 1$, [L4] and [L7] then give $\det G=\det (R^{\mathsf{T}})\det (\bar{R})=\det R\overline{\det R}=\mathrm{\mid }\det R{\mathrm{\mid }}^2$, and $\det R={\prod }_j{R}_{jj}$ is nonzero, so [L5] makes this positive.

If $r=0$, the empty list is independent by [L3] and [L1] gives determinant $1$, which is positive, so all three assertions hold. If $r\ge 1$, steps 1.1 and 2.1 exhaust the dependent and independent cases and give all three assertions.