The Total Derivative in ${\mathbb{R}}^m\to {\mathbb{R}}^n$

Let $m,n\ge 1$. A map $L:{\mathbb{R}}^m\to {\mathbb{R}}^n$ is linear when

$L(\alpha u+\beta v)=\alpha L(u)+\beta L(v)$

for all $u,v\in {\mathbb{R}}^m$ and all $\alpha ,\beta \in \mathbb{R}$. Both spaces carry their Euclidean vector-space operations and Euclidean norms from The Euclidean inner product $⟨x,y⟩={\sum }_{k<n}{x}_k{y}_k$ on ${\mathbb{R}}^n$.

This is the concrete Euclidean notion required for total differentiation. It makes no assertion about linear maps between arbitrary vector spaces.

For every linear $L:{\mathbb{R}}^m\to {\mathbb{R}}^n$ there is a unique matrix $A=(a_{ij}{)}_{i<n,j<m}$ such that $(Lh{)}_i={\sum }_{j<m}{a}_{ij}{h}_j$. Moreover there is $K\ge 0$ with $\mathrm{\parallel }Lh{\mathrm{\parallel }}_2\le K\mathrm{\parallel }h{\mathrm{\parallel }}_2$ for every $h$.

A subset $U\subseteq {\mathbb{R}}^m$ is convex when, for all $x,y\in U$ and $t\in [0,1]$ (Intervals of $\mathbb{R}$: the nine order-convex forms, nondegeneracy, and length), the point $(1-t)x+ty$ lies in $U$. Thus the full line segment from $x$ to $y$ remains in $U$.

Let $U\subseteq {\mathbb{R}}^m$, $a\in U$, $v\in {\mathbb{R}}^m$, and $f:U\to {\mathbb{R}}^n$. If the line map $t\mapsto f(a+tv)$ is defined near $0$, its derivative at $0$ is the directional derivative

$D_{v}f(a):={\lim }_{t\to 0}\frac{f(a+tv)-f(a)}{t}\mathrm{.}$

For a standard basis vector $e_j$ (The standard list $e:n\to F^n$ with $e_i(i)=1_F$ and $e_i(j)=0_F$ for $j\mathrm{\ne }i$ is an ordered basis of $F^n$; hence ${\dim }_F{F}^n=n$, and $F^0$ is the zero space with basis $\mathrm{\varnothing }$ and dimension $0$), $D_{e_j}f(a)$ is the $j$th partial derivative, written ${\mathrm{\partial }}_{j}f(a)$. These are ordinary vector-valued one-variable derivatives in the sense of The derivative and the Riemann integral of a vector-valued function: an intrinsic derivative and a componentwise integral.

Let $U\subseteq {\mathbb{R}}^m$ be open, let $a\in U$, and let $f:U\to {\mathbb{R}}^n$. The map $f$ is totally differentiable at $a$ when there is a linear map $L:{\mathbb{R}}^m\to {\mathbb{R}}^n$ (A linear map $L:{\mathbb{R}}^m\to {\mathbb{R}}^n$ in Euclidean coordinates) such that

${\lim }_{h\to 0}\frac{\mathrm{\parallel }f(a+h)-f(a)-Lh{\mathrm{\parallel }}_2}{\mathrm{\parallel }h{\mathrm{\parallel }}_2}=0,$

where the quotient is considered for $h\mathrm{\ne }0$ with $a+h\in U$. The map $L$, when it exists, is denoted $Df(a)$ and called the total derivative. Equivalently, $f(a+h)=f(a)+Df(a)h+r(h)$ with $\mathrm{\parallel }r(h){\mathrm{\parallel }}_2\mathrm{/}\mathrm{\parallel }h{\mathrm{\parallel }}_2\to 0$.

If $L,M:{\mathbb{R}}^m\to {\mathbb{R}}^n$ both satisfy the total-differentiability remainder condition for $f$ at $a$, then $L=M$.

If $f:U\to {\mathbb{R}}^n$ is totally differentiable at $a\in U$, then some $C,\delta >0$ satisfy $\mathrm{\parallel }f(a+h)-f(a){\mathrm{\parallel }}_2\le C\mathrm{\parallel }h{\mathrm{\parallel }}_2$ whenever $\mathrm{\parallel }h{\mathrm{\parallel }}_2<\delta$ and $a+h\in U$. In particular $f$ is continuous at $a$.

If every partial derivative ${\mathrm{\partial }}_j{f}_i(a)$ of $f:U\to {\mathbb{R}}^n$ exists, the Jacobian matrix is $Jf(a)=({\mathrm{\partial }}_j{f}_i(a){)}_{i<n,j<m}$. For scalar-valued $f$, its gradient is

$\mathrm{\nabla }f(a):=({\mathrm{\partial }}_{0}f(a),\dots ,{\mathrm{\partial }}_{m-1}f(a))\in {\mathbb{R}}^m,$

with coordinates understood in the standard basis (The standard list $e:n\to F^n$ with $e_i(i)=1_F$ and $e_i(j)=0_F$ for $j\mathrm{\ne }i$ is an ordered basis of $F^n$; hence ${\dim }_F{F}^n=n$, and $F^0$ is the zero space with basis $\mathrm{\varnothing }$ and dimension $0$). The partial derivatives are those of Directional derivatives and partial derivatives of a map $U\subseteq {\mathbb{R}}^m\to {\mathbb{R}}^n$.

If $f:U\to {\mathbb{R}}^n$ is totally differentiable at $a$, then $D_{v}f(a)$ exists for every $v\in {\mathbb{R}}^m$ and equals $Df(a)v$. In particular ${\mathrm{\partial }}_{j}f(a)=Df(a)e_j$, and the matrix of $Df(a)$ is $Jf(a)$.

If a scalar-valued $f:U\to \mathbb{R}$ is totally differentiable at $a$, then $D_{v}f(a)=⟨\mathrm{\nabla }f(a),v⟩$ for every $v$. Among unit vectors $v$, this is at most $\mathrm{\parallel }\mathrm{\nabla }f(a){\mathrm{\parallel }}_2$; if the gradient is nonzero, equality holds exactly in the direction $\mathrm{\nabla }f(a)\mathrm{/}\mathrm{\parallel }\mathrm{\nabla }f(a){\mathrm{\parallel }}_2$. If the gradient is zero, every unit direction has directional derivative zero.

If $f,g:U\to {\mathbb{R}}^n$ are totally differentiable at $a$ and $c\in \mathbb{R}$, then $f+g$ and $cf$ are totally differentiable at $a$, with

$D(f+g)(a)=Df(a)+Dg(a),D(cf)(a)=cDf(a)\mathrm{.}$

Let $f:U\to V\subseteq {\mathbb{R}}^n$ be totally differentiable at $a\in U$ and let $g:V\to {\mathbb{R}}^p$ be totally differentiable at $f(a)$. Then $g\circ f$ is totally differentiable at $a$ and

$D(g\circ f)(a)=Dg(f(a))\circ Df(a)\mathrm{.}$

Let $B(a,r)\subseteq U\subseteq {\mathbb{R}}^m$ and let $\mathrm{\parallel }h{\mathrm{\parallel }}_2<r$. Define $p_0=a$ and $p_j=a+{\sum }_{i<j}{h}_i{e}_i$ for $1\le j\le m$. Then every $p_j$ lies in $B(a,r)$, $p_m=a+h$, and for every map $f$ on $U$,

$f(a+h)-f(a)={\sum }_{j<m}(f(p_{j+1})-f(p_j))\mathrm{.}$

Let $U\subseteq {\mathbb{R}}^m$ be open and let $f:U\to {\mathbb{R}}^n$. Suppose every partial derivative ${\mathrm{\partial }}_{j}f$ exists on a neighbourhood of $a\in U$ and is continuous at $a$. Then $f$ is totally differentiable at $a$, and $Df(a)$ is the linear map with matrix $Jf(a)$.

Let $U\subseteq {\mathbb{R}}^m$ be convex and open, and let $f:U\to {\mathbb{R}}^n$ be totally differentiable at every point. If $M\ge 0$ satisfies $\mathrm{\parallel }Df(z)v{\mathrm{\parallel }}_2\le M\mathrm{\parallel }v{\mathrm{\parallel }}_2$ for every $z\in U$ and $v\in {\mathbb{R}}^m$, then

$\mathrm{\parallel }f(y)-f(x){\mathrm{\parallel }}_2\le M\mathrm{\parallel }y-x{\mathrm{\parallel }}_2(x,y\in U)\mathrm{.}$

Let $U\subseteq {\mathbb{R}}^m$ be convex and open. If $f:U\to {\mathbb{R}}^n$ is totally differentiable at every point and $Df(z)=0$ for every $z\in U$, then $f$ is constant on $U$.

The derivative definition is stated on open Euclidean domains so every sufficiently small increment is admissible. Its remainder uses the Euclidean norm; in finite-dimensional Euclidean spaces an equivalent norm would give the same differentiability notion, but that change is not part of this definition.

The linear-map definition on this page is deliberately the concrete Euclidean special case identified in Conventions of this page, the standing $n\ge 1$ hypothesis, and what is taken up elsewhere in the reading order. A future general linear-map development must prove agreement with A linear map $L:{\mathbb{R}}^m\to {\mathbb{R}}^n$ in Euclidean coordinates, not silently replace the meaning of $Df(a)$.