Constant Rank, Submersions, Immersions and Regular Level Sets: Examples and Counterexamples

Given: A radius $R>0$ and $F(x)=⟨x,x⟩$ on ${\mathbb{R}}^m$, with $m\ge 1$.

The Euclidean norm satisfies $\mathrm{\parallel }x{\mathrm{\parallel }}_2^2=⟨x,x⟩$ (The $p$-norms $\parallel x{\parallel }_p$ for rational $p\ge 1$, and $\parallel x{\parallel }_{\mathrm{\infty }}$, The Euclidean inner product $⟨x,y⟩={\sum }_{k<n}{x}_k{y}_k$ on ${\mathbb{R}}^n$). The power rule and one-variable derivative algebra give the continuous Jacobian row $JF(x)=2x^{\mathsf{T}}$, and continuous partial derivatives imply $DF(x)h=2⟨x,h⟩$ (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$, The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative).

A regular level is locally a graph, its tangent space is the derivative kernel, and its tangent vectors are exactly its curve velocities (A regular level set is locally a $C^k$ graph of dimension $m-n$, The tangent space to a regular level set, Tangent vectors to a regular level set are exactly its curve velocities).

Given: A symmetric positive-definite matrix $A$ and the quadratic function $F$.

Positive definiteness means $⟨Ax,x⟩>0$ for every $x\mathrm{\ne }0$ (Positive definite, negative definite, semidefinite, and indefinite quadratic forms, The Euclidean inner product $⟨x,y⟩={\sum }_{k<n}{x}_k{y}_k$ on ${\mathbb{R}}^n$).

The power and product rules give continuous partial derivatives for $F(x)=⟨Ax,x⟩$; the Jacobian and continuous-partials theorem therefore give $DF(x)h=⟨Ah,x⟩+⟨Ax,h⟩=2⟨Ax,h⟩$ for symmetric $A$ (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$, The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative).

At a regular level point, the level is locally a graph and its tangent space is the derivative kernel (A regular level set is locally a $C^k$ graph of dimension $m-n$, The tangent space to a regular level set).

Given: The map $\psi$ and the associated map $G$.

Finite sums and scalar multiples of $C^k$ Euclidean maps are $C^k$, coordinate maps are $C^k$ componentwise, and total-derivative algebra gives $DG(x,y)(v,w)=w-D\psi (x)v$ ($C^k$ Euclidean maps are closed under componentwise algebra and composition, $C^k$ Euclidean maps and diffeomorphisms, Sums and scalar multiples of totally differentiable maps are totally differentiable with the expected derivatives).

A regular level is locally a graph and has tangent space equal to the derivative kernel (A regular level set is locally a $C^k$ graph of dimension $m-n$, The tangent space to a regular level set).

Given: The polynomial $F(x,y,z)=x^2+y^2-z^2$.

The power rule and derivative algebra give the continuous Jacobian row $(2x,2y,-2z)$, and the continuous-partials theorem makes $DF(x,y,z)(h)=2x{h}_1+2y{h}_2-2z{h}_3$ (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$, The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative).

Every positive real has a unique positive square root (Existence and uniqueness of $n$-th roots: a unique $a^{1\mathrm{/}n}\ge 0$ with $(a^{1\mathrm{/}n}{)}^n=a$); a regular level is locally a graph with tangent space equal to the derivative kernel (A regular level set is locally a $C^k$ graph of dimension $m-n$, The tangent space to a regular level set).

Given: A natural number $n\ge 1$, the entrywise identifications above, and the map $f$ with components $f_{ij}(A)={\sum }_{k<n}{a}_{ki}{a}_{kj}$ for $i\le j$.

Each component $f_{ij}$ is a polynomial in the entries of $A$, so its partial derivatives of every order exist and are again polynomials, hence continuous (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$). A map each of whose components is $C^k$ for every $k$ is $C^{\mathrm{\infty }}$ ($C^k$ Euclidean maps and diffeomorphisms), and a map whose partial derivatives exist near a point and are continuous there is totally differentiable there (If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative).

If $f$ is totally differentiable at $A$, then the directional derivative $D_{H}f(A)$ exists for every $H$ and equals $Df(A)H$ (A total derivative computes every directional derivative, and its matrix is the Jacobian).

A $C^1$ map is a submersion at a point when its derivative there is surjective, and a value is regular when every point of its fibre is a submersion point (Submersions and immersions between Euclidean open sets, Regular and critical points, regular and critical values, and level sets).

Near each of its points a regular level set of a $C^k$ map $U\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^N$ is a $C^k$ graph of dimension $m-N$, and its tangent space at such a point is the kernel of the derivative (A regular level set is locally a $C^k$ graph of dimension $m-n$, The tangent space to a regular level set).

A linear map is injective exactly when its kernel is trivial, and for a linear map on a finite-dimensional space the dimension of the space is the sum of the dimensions of the kernel and the image (The kernel and image are linear subspaces, and a linear map is injective if and only if its kernel is trivial, Rank-nullity: ${\dim }_{F}V=\mathrm{nullity}T+\mathrm{rank}T$).

Given: The smooth map $F:{\mathbb{R}}^3\to \mathbb{R}$, $F(x,y,z)=x^2+y^2-z^2$.

The power rule and derivative algebra give the continuous Jacobian row $(2x,2y,-2z)$, which is the total derivative by the continuous-partials theorem, and polynomial coordinate expressions are smooth (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$, The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative, $C^k$ Euclidean maps are closed under componentwise algebra and composition).

A point is regular exactly when its derivative is surjective, and the regular-level graph theorem requires that hypothesis (Regular and critical points, regular and critical values, and level sets, A regular level set is locally a $C^k$ graph of dimension $m-n$).

Given: The polynomial $F(x,y)=y^2-x^3$ and the curve $\gamma (t)=(t^2,t^3)$.

The power rule and derivative algebra give the continuous Jacobian row $(-3x^2,2y)$ and ${\gamma }^{\mathrm{\prime }}(t)=(2t,3t^2)$; the continuous-partials theorem identifies that row with $DF(x,y)$ (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$, The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative, The derivative and the Riemann integral of a vector-valued function: an intrinsic derivative and a componentwise integral).

Regularity means surjectivity of the derivative, and a regular level is locally a $C^1$ graph (Regular and critical points, regular and critical values, and level sets, A regular level set is locally a $C^k$ graph of dimension $m-n$).

Given: The polynomial map $f:{\mathbb{R}}^2\to {\mathbb{R}}^2$, $f(x,y)=(x,xy)$.

The power and product rules give the continuous partial derivatives $Jf(x,y)=\left(\begin{array}{cc}1& 0\\ y& x\end{array}\right)$, and the continuous-partials theorem identifies this Jacobian with the total derivative (The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case, For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative).

A square matrix has rank $2$ exactly when its determinant is nonzero, while its nonzero first row gives rank at least $1$; every point of an open set has a ball contained in it (A matrix has rank at least $r$ exactly when it has a nonzero $r$-rowed minor, The rank of a derivative and constant-rank Euclidean maps, The metric topology: a set is open when every one of its points has a ball around it inside the set; closed means open complement).

Given: The polynomial map $F:{\mathbb{R}}^2\to \mathbb{R}$, $F(x,y)=x^2$.

The power rule gives the continuous Jacobian row $(2x,0)$, and the continuous-partials theorem identifies it with $DF(x,y)$ (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative).

A value is critical when some point of its fibre has nonsurjective derivative (Regular and critical points, regular and critical values, and level sets), while the graph of a $C^1$ map is a regular level set for a suitable defining map (The graph of a $C^k$ Euclidean map is a regular level set).

Given: A natural $m\ge 1$, the radius $R>0$, vector $a\in {\mathbb{R}}^m$, objective $f(x)=⟨a,x⟩$, and constraint $G(x)=\mathrm{\parallel }x{\mathrm{\parallel }}_2^2-R^2$.

The sphere constraint is regular, and the one-constraint multiplier rule gives $\mathrm{\nabla }f(x)=\lambda \mathrm{\nabla }G(x)$ at every constrained local extremum (A Euclidean sphere is a regular level set with tangent hyperplanes, For one regular constraint, the objective gradient is a scalar multiple of the constraint gradient).

Cauchy-Schwarz gives $\mathrm{\mid }⟨a,x⟩\mathrm{\mid }\le \mathrm{\parallel }a{\mathrm{\parallel }}_2\mathrm{\parallel }x{\mathrm{\parallel }}_2$, with equality precisely for linearly dependent vectors (Cauchy-Schwarz $\mid ⟨x,y⟩\mid \le \parallel x{\parallel }_2\parallel y{\parallel }_2$ with its equality case, the triangle inequality for $\parallel \cdot {\parallel }_2$, the parallelogram law and polarisation, The Euclidean inner product $⟨x,y⟩={\sum }_{k<n}{x}_k{y}_k$ on ${\mathbb{R}}^n$).

Given: The maps $f(x,y,z)=xy+xz+yz$ and $G(x,y,z)=(x^2+y^2+z^2,z)$ on ${\mathbb{R}}^3$, the value $c=(2,1)$, and $M=G^{-1}(c)$. Extrema on $M$ are constrained extrema, comparing $f$ only at points of $M$: a point $a\in M$ is a local extremum of $f$ on $M$ when for some $r>0$ either $f(p)\le f(a)$ for every $p\in M$ with $\parallel p-a{\parallel }_2<r$, or $f(p)\ge f(a)$ for every such $p$; it is a strict local maximum of $f$ on $M$ when for some $r>0$, $f(p)<f(a)$ for every $p\in M$ with $0<\parallel p-a{\parallel }_2<r$. This is the sense of "local maximum or minimum of $f$ subject to $G(x)=c$" in [L3]; [L4] is the unconstrained notion, comparing $f$ at every nearby point of the open set on which it is defined.

A polynomial in one real variable is differentiable with the derivative computed by the power, sum and product rules, so each partial derivative of $f$ and of the components of $G$ is again a polynomial and hence continuous; a map whose partial derivatives exist near a point and are continuous there is totally differentiable there, with derivative the Jacobian matrix (For a natural $n\ge 1$ the function $x\mapsto x^n$ is differentiable everywhere with derivative $\iota (n)x^{n-1}$; for $n=0$ it is the constant $1$, with derivative $0$; for a natural $n\ge 1$ the function $x\mapsto x^{-n}$ is differentiable at every $x\mathrm{\ne }0$ with derivative $-\iota (n)x^{-n-1}$; consequently every polynomial function is differentiable at every real, with the derivative computed term by term, Sums, scalar multiples, products and quotients: $(f+g{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)+g^{\mathrm{\prime }}(c)$, $(\alpha f{)}^{\mathrm{\prime }}(c)=\alpha f^{\mathrm{\prime }}(c)$, $(fg{)}^{\mathrm{\prime }}(c)=f^{\mathrm{\prime }}(c)g(c)+f(c)g^{\mathrm{\prime }}(c)$, and $(f\mathrm{/}g{)}^{\mathrm{\prime }}(c)=(f^{\mathrm{\prime }}(c)g(c)-f(c)g^{\mathrm{\prime }}(c))\mathrm{/}g(c{)}^2$ when $g(c)\mathrm{\ne }0$, If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative, The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case).

A $C^1$ map is a submersion at a point when its derivative there is surjective, and a matrix has rank at least $r$ exactly when some $r$-rowed minor is nonzero (Submersions and immersions between Euclidean open sets, A matrix has rank at least $r$ exactly when it has a nonzero $r$-rowed minor).

If $a$ is a local maximum or minimum of a $C^1$ objective $f$ subject to $G(x)=c$ with $G$ of class $C^1$, and $DG(a)$ is surjective, then there is a unique $\lambda$ with $\mathrm{\nabla }f(a)=DG(a{)}^{\mathsf{T}}\lambda$; the condition is necessary and not sufficient (Lagrange multipliers for a regular vector-valued level-set constraint).