Conjugacy of diagonalizable complements and maximal subgroups under smoothness hypotheses

First suppose $D$ is smooth. Induct on the length of [F1], with $U=1$ as base. Let $N$ be its last nontrivial term, and compare two sections in $G\mathrm{/}N$. By [F2] induction makes those quotient sections conjugate by $(U\mathrm{/}N)(k)$. Lift the conjugating point to $U(k)$ using [F9] and conjugate one original section so their quotient sections agree. Their ratio is a crossed homomorphism $f:D\to N$ for the actual linear action on the additive embedding. Reducedness of $D$ makes $f$ factor through $N_{\mathrm{r}\mathrm{e}\mathrm{d}}$. The reduction is $D$-stable because $D\times N_{\mathrm{r}\mathrm{e}\mathrm{d}}$ is reduced, and is smooth commutative unipotent by [F3]. The principal-cocycle theorem [F3] therefore gives a conjugating point of $N_{\mathrm{r}\mathrm{e}\mathrm{d}}(k)=N(k)$. This completes induction and proves (a) for smooth $D$, with arbitrary $U$.

For the second domain of (a), establish $H^1(D,N)=0$ for every vector group $N$ with any action of diagonalizable $D$. In characteristic zero its additive automorphisms are linear, so [F7] applies directly. In characteristic $p$, use [F8]: decompose the coordinate primitives of $P(N)=B^n$ into finitely many $D$-weight components, and take a free $B$-module $L={⨁}_{i}B{e}_i$ on these homogeneous generators. The kernel $R$ of $L\to P(N)$ has a homogeneous free basis. Indeed choose a homogeneous relation of smallest nonzero first-coordinate $F$-degree; Euclidean reduction of another homogeneous relation uses multiples $a{F}^j$ with the same leading-coordinate weight, so each subtraction remains homogeneous. A nonzero lower-degree remainder contradicts minimality. This splits off one free pivot summand, and repeat on the zero-first-coordinate kernel and the remaining coordinates. The process terminates and handles arbitrary relations by their finite weight decompositions. The reversed exact sequence from [F8] is $0\to N\to V\stackrel{f}{\to }Q\to 0$, where the homogeneous bases make $V,Q$ linear vector representations. Since $N$ is a vector group as an underlying group, $f$ is a smooth vector-group torsor and its pullbacks to affine $D^j$ are trivial by [F7]. Hence this coefficient sequence is exact on every represented cochain, including nonreduced $D^j$.

Suppose $G$ is smooth. Its quotient $D$ is smooth by [F9], so $D^{\circ }$ is a torus. Let $U_0$ be the largest normal unipotent subgroup of the smooth connected trigonalizable group $G^{\circ }$; it is smooth connected by [F9]. Conjugation by every $g\in G(k)$ preserves $G^{\circ }$ and its unique maximal normal unipotent subgroup. Since $G\times U_0$ is smooth with schematically dense $k$-points, this pointwise preservation gives scheme-theoretic normality of $U_0$ in $G$. Thus $U_0\subseteq U$. Conversely $U\cap G^{\circ }$ is a normal unipotent subgroup of $G^{\circ }$, so it is contained in $U_0$; hence $U\cap G^{\circ }=U_0$. This intersection is open and closed in $U$ and connected, so $U^{\circ }=U_0$. The faithfully flat quotient $q$ is of finite presentation and is open by [F9]. Thus $q(G^{\circ })$ is an open connected subgroup of $D$, contained in $D^{\circ }$; an open subgroup of the connected group $D^{\circ }$ is all of it, since its cosets would otherwise disconnect that group. Consequently $q(G^{\circ })=D^{\circ }$, and $G^{\circ }\to D^{\circ }$ has kernel $U_0$.

The limitations are explicit in characteristic $p>0$. In $G={\alpha }_p⋊{\mu }_p$ with scalar action, $U={\alpha }_p$ and $D={\mu }_p$. For every $a\in k$, $s_a(t)=(a(t-1),t)$ is a section: $a(t-1)$ has $p$-th power zero, and $a(t{t}^{\mathrm{\prime }}-1)=a(t-1)+ta(t^{\mathrm{\prime }}-1)$ proves its cocycle identity on all base algebras. Distinct $a$ give distinct sections, but $U(k)=\{0\}$, so they are not $U(k)$-conjugate. In $G={\alpha }_p⋊{\mathbf{G}}_m$, $D={\mathbf{G}}_m$ is smooth and its unique section is $s_0$: a morphism from reduced ${\mathbf{G}}_m$ into ${\alpha }_p$ is zero. Nevertheless $S_a=\{(a(t-1),t):t\in {\mu }_p\}$ for $a\mathrm{\ne }0$ is a nonsmooth diagonalizable subgroup not contained in $s_0(D)$. It is maximal diagonalizable: any larger such subgroup has image either ${\mathbf{G}}_m$ or ${\mu }_n\subseteq {\mathbf{G}}_m$, by the character anti-equivalence. The first option would be the unique full section. A section over ${\mu }_n$ for the weight-one action has cocycle $b(t-1)$, by coefficient comparison in $k[\mathbb{Z}\mathrm{/}n\mathbb{Z}]$; containing $S_a$ forces $b=a$. Its image lies in ${\alpha }_p$ only if $t^p=1$ in $O({\mu }_n)$, hence $n$ divides $p$, and containment of ${\mu }_p$ gives $n=p$. Thus $S_a$ has no larger diagonalizable overgroup. Both groups are trigonalizable by their unipotent kernels and diagonalizable quotients, and splitting still exists. These examples refute the unrestricted claims but not the domains proved above.

The vector-group torsor $f:V\to Q$ has a scheme section by [F7]; translate its value at zero by an element of $N(k)$ to obtain a section taking zero to zero. Taking degree-one terms in $f\circ s={\mathrm{id}}_Q$ proves that $d{f}_0$ is surjective. Its kernel is the tangent space of the scheme kernel $N$, so $0\to \mathrm{Lie}N\to \mathrm{Lie}V\to \mathrm{Lie}Q\to 0$ is exact. Invariants are exact for diagonalizable representations by weight decomposition, so the tangent map $\mathrm{Lie}(V^D)\to \mathrm{Lie}(Q^D)$ is surjective; here $V^D,Q^D$ are the weight-zero linear vector subgroups. The additive-polynomial map $V^D\to Q^D$ therefore has a Jacobian of full target rank, constant under translation. Its graph presentation over $k[y_1,\dots ,y_b]$ has equations $y_i-P_i(x)$ with an invertible $b\times b$ minor in the $x$-Jacobian (the empty minor if $b=0$). The Jacobian criterion Relative Jacobian criterion with its presentation hypothesis applied to this finite-presentation graph proves smoothness; hence it is open by Flat finite-presentation morphisms are open. Its image is an open subgroup of the connected vector group $Q^D$, hence all of $Q^D$ (otherwise its cosets give a disconnection). Its nonempty finite-type fibres have $k$-points by [F9], so $V^D(k)\to Q^D(k)$ is surjective. In the long exact cochain sequence, $H^0(D,V)\to H^0(D,Q)\to H^1(D,N)\to H^1(D,V)$ consequently has surjective first map and zero last term by [F7]. Thus $H^1(D,N)=0$, for arbitrary nonlinear actions and nonsmooth $D$.

Now assume $U$ smooth connected and induct on its dimension. If $U=1$ there is a unique section. Otherwise choose its last nontrivial derived subgroup $A$, smooth connected commutative and characteristic by [F7]. In characteristic zero set $N=A$, a vector group by [F8]. In characteristic $p$, multiplication by large $p$-powers kills $A$ by its upper unitriangular embedding; its last nonzero multiplication image $N=p^{j}A$ is smooth connected by [F7], killed by $p$, and therefore a vector group by [F8]. This $N$ is characteristic in $U$ after every base change, hence normal in $G$, and positive-dimensional. By [F2] and [F7], $G\mathrm{/}N$ has smooth connected unipotent subgroup $U\mathrm{/}N$ of smaller dimension and the same $D$. Induction conjugates the quotient sections by $(U\mathrm{/}N)(k)$; lift that point to $U(k)$ by [F9] and make the quotient sections equal. Their ratio is now a crossed homomorphism $D\to N$, which is principal by step 2.1. Conjugation by its principal point identifies the sections. This proves (a) when $U$ is smooth connected, including nonsmooth $D$.

Fix a full section $s:D\to G$, which exists by [F4]. For a diagonalizable subgroup $S\subseteq G$, [F5] identifies it with its closed image $E=q(S)\subseteq D$. In $q^{-1}(E)$, both $s{\mathrm{\mid }}_E$ and the section supplied by $S$ have unipotent kernel $U$ by [F6]. If $U$ is smooth connected, step3.1 applies even when $E$ is nonsmooth; hence $S$ lies in a $U(k)$-conjugate of $s(D)$. If instead only $D$ is smooth and $S$ is smooth diagonalizable, then $E$ is smooth, and step1.1 gives the same containment. In either domain maximality gives equality with a section image. Conversely, a diagonalizable subgroup containing $s(D)$ equals it: the map to $D$ has trivial kernel, and its points have the same images as the points of $s(D)$ on every base algebra. The same argument applies in the smooth-diagonalizable class when $D$ is smooth. Part(a) gives conjugacy of all relevant section images, proving (b) with the specified domains.

All tori of $G$ lie in $G^{\circ }$. The smooth connected splitting theorem [F4] applies to $G^{\circ }$ and identifies its maximal tori with the images of sections of $G^{\circ }\to D^{\circ }$. The restriction of a fixed full section $s:D\to G$ is one such section, since $s(D^{\circ })\subseteq G^{\circ }$. Every other section over $D^{\circ }$ is $U_0(k)$-conjugate to it by step3.1 (or the smooth connected splitting theorem); conjugating the full $s$ by that same point extends the desired partial section. Hence precisely the groups $s(D^{\circ })$ for full sections are the maximal tori, and they are $U^{\circ }(k)$-conjugate. If $G$ is connected, its quotient is connected, so $D=D^{\circ }$. This proves (c) independently of any full maximal-diagonalizable classification for disconnected $U$.

Steps1.1 and3.1 prove the two domains of(a), step4.1 proves both classifications in(b), and steps 1.3 and 4.2 give the independent smooth-group torus claim(c). Step 1.4 establishes the stated boundaries while retaining arbitrary-scheme splitting existence.