Complete reducibility and the Reynolds operator for a complex reductive group

Treat first connected $G$. By [F2] identify $G$ with a closed algebraic subgroup of $GL(W)$ in a faithful finite-dimensional rational representation. Smoothness [F3] and the Jacobian and holomorphic charts [F16] make $G(\mathbb{C})$ a complex manifold. Regular multiplication, inversion and representation maps are holomorphic in these charts, so it is a complex Lie subgroup of $GL(W)$ with complex tangent algebra $\mathfrak{g}=T_{e}G\subseteq \mathrm{End}(W)$; differentiation preserves brackets by [F18]. Its underlying real Lie group is the closed embedded subgroup of [F5], by uniqueness there. Exponential naturality [F17] and [F4] imply $\exp (tX)\in G(\mathbb{C})$ for $X\in \mathfrak{g}$ and $t\in \mathbb{C}$ (apply the real assertion to $tX$). These curves are analytic; no algebraicity of $t\mapsto \exp (tX)$ is asserted except when $X$ is nilpotent.

Let $E$ be the abstract subgroup generated by $\exp (\mathfrak{g})$ and $H$ its Zariski closure in $G$. The closure of a subgroup is a subgroup: multiplication and inverse preserve it by continuity, first translating by each member of $E$ and then taking closure in the other variable. All complex closed algebraic subgroups are smooth by [F3], and the curves $t\mapsto \exp (tX)$ show $\mathfrak{g}\subseteq T_{e}H$, hence $\dim H=\dim G$. A smooth connected algebraic group is geometrically integral by Connected finite-type groups are geometrically connected, so it is irreducible. Consequently a full-dimensional closed subgroup of connected $G$ equals $G$, and $E$ is Zariski dense. If $U\subseteq V$ is $\mathfrak{g}$-invariant, exponential naturality makes it invariant under $E$. Its algebraic stabilizer is closed, since in a basis adapted to $U$ the lower-left matrix entries of the representation vanish precisely on that stabilizer. It contains dense $E$, so is all $G$. The same argument applies to every member of a flag.

Let $\mathfrak{r}=\mathrm{rad}(\mathfrak{g})$ be the radical and choose by [F6] a basis of $W$ in which every $A\in \mathfrak{r}$ is upper triangular, with diagonal linear forms ${\lambda }_1,\dots ,{\lambda }_N\in {\mathfrak{r}}^{\ast }$. The radical is characteristic by [F7], so conjugation by $h\in G$ preserves $\mathfrak{r}$ and carries the representation $\mathfrak{r}\to \mathrm{End}(W)$ to an equivalent one; the multiset of diagonal values of the conjugate of $A$ is $\{{\lambda }_j(A){\}}_j$, so each composition ${\lambda }_i\circ \mathrm{Ad}(h)$ is one of ${\lambda }_1,\dots ,{\lambda }_N$ (a value function not among them is separated from all of them by evaluating at some $A$ outside finitely many proper hyperplanes). Each map $h\mapsto {\lambda }_i\circ \mathrm{Ad}(h)$ is a morphism from the connected group $G$ to the finite set $\{{\lambda }_1,\dots ,{\lambda }_N\}$, hence is constant; differentiating at the identity gives ${\lambda }_i([X,A])=0$ for all $X\in \mathfrak{g}$, $A\in \mathfrak{r}$. Consequently the commutator ideal $\mathfrak{n}=[\mathfrak{g},\mathfrak{r}]$ is a $\mathrm{Ad}(G)$-stable Lie ideal of $\mathfrak{g}$ whose elements are strictly upper triangular in this basis.

Let $N=⟨\exp (\mathfrak{n})⟩$ and let $U=\bar{N}$ be its Zariski closure. Every $\exp (tA)$ with $A\in \mathfrak{n}$ strictly upper triangular is a polynomial function of $t$, so $U$ is a closed connected subgroup of $G$ contained in the upper unitriangular group; it is normal, because $\mathrm{Ad}(G)\mathfrak{n}=\mathfrak{n}$ and conjugation commutes with the exponential. To check unipotence in the fixed-vector sense of the definition, let $M$ be a non-zero finite-dimensional rational $U$-module: its Lie algebra $\mathfrak{u}$ is solvable, so [F6] gives a complete $\mathfrak{u}$-invariant flag, which by step 2.1 is $U$-invariant, and every diagonal character of $U$ is trivial because on each $\exp (tA)$ it restricts to an algebraic homomorphism $\mathbb{C}\to {\mathbb{C}}^{\times }$ whose coordinate function and inverse are both polynomial, hence constant; the first line of the flag is therefore a non-zero fixed vector. If $\mathfrak{n}\mathrm{\ne }0$ then $U\mathrm{\ne }1$, since the matrix exponential is injective on nilpotent matrices, and reductivity of $G$ forces $\mathfrak{n}=0$.

Since $[\mathfrak{g},\mathfrak{r}]=\mathfrak{n}=0$ by step 3.1, the radical is central: every $A\in \mathfrak{r}$ commutes with $\mathfrak{g}$. Hence $\mathfrak{r}\subseteq \mathfrak{z}(\mathfrak{g})$; conversely the centre is abelian and therefore solvable, so it lies in the radical, giving $\mathfrak{r}=\mathfrak{z}(\mathfrak{g})$. By [F8] the Lie algebra splits as $\mathfrak{g}=\mathfrak{r}⋊\mathfrak{s}=\mathfrak{z}(\mathfrak{g})\oplus \mathfrak{s}$ with $\mathfrak{s}\cong \mathfrak{g}\mathrm{/}\mathfrak{r}$ semisimple, its radical being zero by [F7].

Every $X\in \mathfrak{z}(\mathfrak{g})$ is semisimple as a matrix. First, $\exp (tX)$ is central in $G$ for every $t$: the centralizer of $X$ in $G$ is closed and its Lie algebra contains $\mathfrak{g}$, so step 2.1 applied to the adjoint representation gives centrality. Write the additive Jordan decomposition $X=X_s+X_n$ of [F9] over the perfect field $\mathbb{C}$; both parts are polynomials in $X$, hence commute with every element of $G$. For a regular function $P$ on $GL(W)$ vanishing on $\exp (tX)$, substituting the exponential writes $P(\exp (tX))={\sum }_{\mu }{p}_{\mu }(t)e^{\mu t}$ with polynomials $p_{\mu }$, since determinant denominators contribute only further exponentials; the distinct functions $e^{\mu t}$ are linearly independent over $\mathbb{C}[t]$ because applying ${\prod }_{\nu \mathrm{\ne }{\mu }_0}(D-\nu {)}^{1+\mathrm{deg}{p}_{\nu }}$ to a relation kills all other terms and leaves a non-zero polynomial multiple of $e^{{\mu }_{0}t}$. Hence all $p_{\mu }$ vanish, and substituting $\exp (t{X}_s)\exp (u{X}_n)$ gives ${\sum }_{\mu }{p}_{\mu }(u)e^{\mu t}=0$ for independent $t,u$, so every such $P$ vanishes on $\exp (u{X}_n)$. Therefore $\exp (\mathbb{C}{X}_n)$ lies in the Zariski closure of $\exp (\mathbb{C}X)$, which is central in $G$. If $X_n\mathrm{\ne }0$, the map $u\mapsto \exp (u{X}_n)$ is injective with closed image isomorphic to ${\mathbb{G}}_a$ (the finite polynomial $\log (1+M)$ is its inverse and recovers $u$ from an entry), and by the flag argument of step 3.1 the group ${\mathbb{G}}_a$ has a non-zero fixed vector in every non-zero rational module, so it would be a non-trivial closed normal unipotent subgroup of $G$, contradicting reductivity. Hence $X_n=0$ and $X$ is semisimple.

The family $\mathfrak{z}(\mathfrak{g})$ consists of commuting semisimple endomorphisms, so by [F10] the space $W$ decomposes as ${⨁}_{\chi }{W}_{\chi }$ with each $X\in \mathfrak{z}(\mathfrak{g})$ acting on $W_{\chi }$ by the character $\chi$. Let $T=\overline{⟨\exp (\mathfrak{z}(\mathfrak{g}))⟩}$; by step 5.1 this is contained in the diagonal torus of $GL(W)$, is central in $G$, and its Lie algebra contains $\mathfrak{z}(\mathfrak{g})$. The restrictions to the closed subgroup $T$ of Laurent monomials in the diagonal coordinates span $\mathbb{C}[T]$, because the coordinate-ring restriction from the diagonal torus is surjective; these monomials restrict to characters. Distinct characters of a group are linearly independent (a shortest non-trivial relation evaluated at $hg$ and compared with its translate by $\chi (h)$ yields a shorter one), so the restricted characters form a basis of $\mathbb{C}[T]$; for a finite-dimensional rational $T$-module the coaction expansion in this basis exhibits the module as a direct sum of weight spaces, and coassociativity with $\mathrm{\Delta }\chi =\chi \otimes \chi$ shows each coefficient is a weight vector whose sum is the original vector. These weight spaces are $G$-stable because $T$ is central, and the Lie algebra $\mathfrak{z}(\mathfrak{g})$ acts scalarly on each of them. Given a $G$-stable subspace $U$ of a finite-dimensional rational $G$-module $V$, decompose $V$ into $T$-weight spaces, use [F11] to split the semisimple Lie algebra $\mathfrak{s}$ on each weight space and obtain an $\mathfrak{s}$-stable complement of $U$ there; the scalar action of $\mathfrak{z}(\mathfrak{g})$ makes these complements $\mathfrak{g}$-stable, and step 2.1 makes their direct sum $G$-stable. This proves (i) for connected $G$.

For possibly disconnected reductive $G$, the identity component $G^{\circ }$ is reductive: if $U_0\mathrm{\ne }1$ were a closed normal unipotent subgroup of $G^{\circ }$, its finitely many $G$-conjugates $U_1,\dots ,U_m$ would be normal in $G^{\circ }$ (conjugation permutes them), and a common fixed vector for $U_1,\dots ,U_m$ on any non-zero finite-dimensional rational module exists by successively restricting and using normality in the fixed-vector definition; hence the closed subgroup generated by the conjugates is a non-trivial closed normal unipotent subgroup of $G$, contradicting reductivity. So step 6.1 applies to $G^{\circ }$. The quotient $G\mathrm{/}{G}^{\circ }$ is finite: the cosets $g{G}^{\circ }$ are disjoint open sets covering the quasi-compact space $G$, so finitely many suffice. If $U\subseteq V$ is a $G$-stable subspace, choose a $G^{\circ }$-equivariant projection $p:V\to U$ (complete reducibility for $G^{\circ }$) and average $q=\frac{1}{m}{\sum }_i{h}_{i}p{h}_i^{-1}$ over representatives $h_1,\dots ,h_m$ of $G\mathrm{/}{G}^{\circ }$; each conjugate is again a $G^{\circ }$-equivariant projection onto $U$, and $q$ is $G$-equivariant with kernel a $G$-stable complement of $U$. Hence (i) holds for $G$, and the averaging uses only a finite choice of representatives.

For (ii), let $V$ be any rational $G$-module. Every vector of $V$ lies in a finite-dimensional $G$-stable submodule by [F1], which decomposes as a finite direct sum of simples by step 7.1, so $V$ is the sum of its simple submodules; under AC, [F12] upgrades this to a direct sum decomposition. Let $V_G$ be the sum of all simple submodules on which $G$ acts non-trivially. A non-trivial simple module $S$ has no non-zero morphism to the trivial module: the image would be a non-zero simple submodule of the trivial module, hence the whole of it, and the kernel would be a proper submodule of $S$, hence zero, making $S$ trivial. Therefore the trivial isotypic part is exactly $V^G$, the submodule $V_G$ is a complement of it, and every $G$-stable complement of $V^G$ contains no trivial simple submodule, hence equals $V_G$; the projection $R_V$ along $V_G$ is therefore canonical. It is $G$-equivariant, fixes $V^G$ pointwise and is natural: an equivariant $f$ maps trivial simples to trivial simples and non-trivial simples to non-trivial ones, so $f(V_G)\subseteq W_G$ and $R_{W}f=f^G{R}_V$. If $f$ is surjective and $w\in W^G$, lift $w=f(v)$ and compute $w=R_W(w)=R_W(fv)=f(R_{V}v)$, so $f^G$ is surjective. For (iii), multiplication by $a\in A^G$ is an equivariant endomorphism of $A$, so naturality gives $R_A(ab)=a{R}_A(b)$; this completes (ii) and (iii).

The converse recorded in the definition also holds, without using the omitted compact-existence direction. If every finite-dimensional rational $G$-module is completely reducible and $U\subseteq G$ is a closed normal unipotent subgroup, take a faithful finite-dimensional module $W$ from [F2] and a simple submodule $S\subseteq W$; by the fixed-vector definition of unipotence $S^U\mathrm{\ne }0$, and $S^U$ is $G$-stable because $U$ is normal, so $S^U=S$ by simplicity and $U$ fixes every simple summand of $W$, hence all of $W$; faithfulness forces $U=1$. Thus linearly reductive implies reductive. Closed subgroups of a fixed-vector unipotent group $U$ are again unipotent, as follows. A faithful finite-dimensional module for $U$ from [F2] has a complete flag with trivial successive characters, by iterating the fixed-vector condition on its quotients, so $U$ is a closed subgroup of an upper unitriangular matrix group. For a closed subgroup $H\subseteq U$ and $h\in H$, the nilpotent matrix $\log h$ is a finite polynomial in $h-1$ and $t\mapsto \exp (t\log h)$ is polynomial. Every defining equation of $H$ vanishes on this curve at all nonnegative integers because these values are $h^n$, so it vanishes identically; the curve lies in $H$ and joins $1$ to $h$. Thus $H$ is connected. The Lie algebra of $H$ is strictly upper triangular, hence solvable. For any nonzero rational $H$-module, [F6] gives a Lie-invariant complete flag, which is $H$-invariant by step 2.1. Its diagonal characters are trivial on each curve $\exp (t\log h)$: an algebraic homomorphism ${\mathbb{G}}_a\to {\mathbb{G}}_m$ has polynomial coordinate and polynomial inverse, so is constant. Since every $h\in H$ lies on such a curve, the first flag line is fixed by $H$, proving the hereditary assertion. For the identity-component reductions: $G$ is reductive if and only if $G^{\circ }$ is by the conjugate-product argument of step 7.1 in one direction. For the other direction, if $G^{\circ }$ is reductive and $U$ is a normal unipotent subgroup of $G$, then $U\cap G^{\circ }=1$, so $U$ embeds in the finite group $G\mathrm{/}{G}^{\circ }$. A nontrivial finite group over $\mathbb{C}$ is not unipotent: its regular representation has the nonzero augmentation submodule, on which the only possible invariant vectors are multiples of the sum of all basis vectors, and none has augmentation zero in characteristic zero. Hence $U=1$; $G$ is linearly reductive if and only if $G^{\circ }$ is, the forward direction by the implication linearly reductive $G\Rightarrow$ reductive $G\Rightarrow$ reductive $G^{\circ }$ proved above, followed by step 6.1 and the reverse by the finite averaging of step 7.1. Moreover smoothness makes the irreducible components of $G$ disjoint, so they are the connected components and the cosets of $G^{\circ }$; there are finitely many of them.

For (iv), let $K\subseteq G$ be compact with Zariski closure $G$; as a compact subset of the Hausdorff space $G$ it is closed, hence by [F5] a compact Lie subgroup, so [F13] supplies its normalized Haar measure $dg$. For $v\in V$ put $A(v)={\int }_{K}g\cdot vdg$, an integral of a continuous map on the compact group. Translation invariance [F14] gives $h\cdot A(v)=A(v)$ for every $h\in K$, so the algebraic stabilizer of $A(v)$ is a closed subgroup of $G$ containing $K$, hence equal to $G$ because the Zariski closure of $K$ is $G$; thus $A(v)\in V^G$ and $A$ fixes $V^G$ pointwise. The module $V$ is completely reducible as a $K$-module by [F15], so $V=V^K\oplus V_{K\text{-nontriv}}$ with $V_{K\text{-nontriv}}$ the sum of the non-trivial isotypic components; since $V^K=V^G$ by Zariski density, the same simple-module argument as in step 8.1 gives uniqueness of a $K$-stable complement to $V^K$. Since $V_G$ is such a complement, this shows $V_{K\text{-nontriv}}=V_G$, and the Haar average is the $K$-equivariant projection onto $V^K$, that is $A=R_V$. The convex hull of $K\cdot v$ is compact: in the underlying real space of dimension $q$, affine dependence reduces each convex combination to at most $q+1$ terms by subtracting a scalar multiple of an affine relation until a coefficient becomes zero; the hull is therefore the image of the compact product $(K\cdot v{)}^{q+1}$ with the compact coefficient simplex. Finally, the Haar average lies in this closed convex hull of the compact orbit $K\cdot v$ (uniform continuity on the compact group writes it as a limit of finite convex combinations), and for any invariant $w$ in that hull, write $w$ as a limit of finite convex combinations $w_n={\sum }_i{c}_{i,n}{g}_{i,n}v$; linearity, $G$-equivariance of $R_V$ and $R_V(v)\in V^G$ give $R_V(w_n)=R_V(v)$ for every $n$, so continuity of $R_V$ gives $w=R_V(w)=R_V(v)$ whenever $w\in V^G$. Hence $R_V(v)$ is the unique element of $V^G$ in the convex hull of the $K$-orbit of $v$.

Assembling the clauses: (i) is step 6.1 for connected $G$ and step 7.1 for general reductive $G$, together with the converse implication proved in step 9.1; (ii) and (iii) are step 8.1; (iv) is step 9.2, whose hypothesis on $K$ is part of the clause and whose proof does not assert the existence of such a $K$; and the identity-component and converse statements recorded in the definition are step 9.1. The Axiom of Choice is inherited from the named suppliers: the faithful embedding, smoothness and integrality inputs enter in steps 1.1 and 2.1, the Lie inputs and Weyl's theorem in steps 2.2-7.1, the sum-of-simples characterisation in step 8.1, the faithful module in step 9.1, and the Haar measure and compact complete reducibility in step 9.2. This proves all four clauses.