The Total Derivative in $\mathbb{R}^m \to \mathbb{R}^n$

Let $m,n\ge1$. A map $L:\mathbb R^m\to\mathbb R^n$ is linear when

$L(\alpha u+\beta v)=\alpha L(u)+\beta L(v)$

for all $u,v\in\mathbb R^m$ and all $\alpha,\beta\in\mathbb R$. Both spaces carry their Euclidean vector-space operations and Euclidean norms from The Euclidean inner product $\langle x,y\rangle = \sum_{k<n} x_k y_k$ on $\mathbb{R}^n$.

This is the concrete Euclidean notion required for total differentiation. It makes no assertion about linear maps between arbitrary vector spaces.

For every linear $L:\mathbb R^m\to\mathbb R^n$ there is a unique matrix $A=(a_{ij})_{i<n,j<m}$ such that $(Lh)_i=\sum_{j<m}a_{ij}h_j$. Moreover there is $K\ge0$ with $\|Lh\|_2\le K\|h\|_2$ for every $h$.

A subset $U\subseteq\mathbb R^m$ is convex when, for all $x,y\in U$ and $t\in[0,1]$ (Intervals of $\mathbb{R}$: the nine order-convex forms, nondegeneracy, and length), the point $(1-t)x+ty$ lies in $U$. Thus the full line segment from $x$ to $y$ remains in $U$.

Let $U\subseteq\mathbb R^m$, $a\in U$, $v\in\mathbb R^m$, and $f:U\to\mathbb R^n$. If the line map $t\mapsto f(a+tv)$ is defined near $0$, its derivative at $0$ is the directional derivative

$D_vf(a):=\lim_{t\to0}\frac{f(a+tv)-f(a)}{t}.$

For a standard basis vector $e_j$ (The standard list $e : n \to F^{n}$ with $e_i(i) = 1_F$ and $e_i(j) = 0_F$ for $j \ne i$ is an ordered basis of $F^{n}$; hence $\dim_F F^{n} = n$, and $F^{0}$ is the zero space with basis $\varnothing$ and dimension $0$), $D_{e_j}f(a)$ is the $j$th partial derivative, written $\partial_jf(a)$. These are ordinary vector-valued one-variable derivatives in the sense of The derivative and the Riemann integral of a vector-valued function: an intrinsic derivative and a componentwise integral.

Let $U\subseteq\mathbb R^m$ be open, let $a\in U$, and let $f:U\to\mathbb R^n$. The map $f$ is totally differentiable at $a$ when there is a linear map $L:\mathbb R^m\to\mathbb R^n$ (A linear map $L:\mathbb{R}^m\to\mathbb{R}^n$ in Euclidean coordinates) such that

$\lim_{h\to0}\frac{\|f(a+h)-f(a)-Lh\|_2}{\|h\|_2}=0,$

where the quotient is considered for $h\ne0$ with $a+h\in U$. The map $L$, when it exists, is denoted $Df(a)$ and called the total derivative. Equivalently, $f(a+h)=f(a)+Df(a)h+r(h)$ with $\|r(h)\|_2/\|h\|_2\to0$.

If $L,M:\mathbb R^m\to\mathbb R^n$ both satisfy the total-differentiability remainder condition for $f$ at $a$, then $L=M$.

If $f:U\to\mathbb R^n$ is totally differentiable at $a\in U$, then some $C,\delta>0$ satisfy $\|f(a+h)-f(a)\|_2\le C\|h\|_2$ whenever $\|h\|_2<\delta$ and $a+h\in U$. In particular $f$ is continuous at $a$.

If every partial derivative $\partial_jf_i(a)$ of $f:U\to\mathbb R^n$ exists, the Jacobian matrix is $Jf(a)=(\partial_jf_i(a))_{i<n,j<m}$. For scalar-valued $f$, its gradient is

$\nabla f(a):=(\partial_0f(a),\ldots,\partial_{m-1}f(a))\in\mathbb R^m,$

with coordinates understood in the standard basis (The standard list $e : n \to F^{n}$ with $e_i(i) = 1_F$ and $e_i(j) = 0_F$ for $j \ne i$ is an ordered basis of $F^{n}$; hence $\dim_F F^{n} = n$, and $F^{0}$ is the zero space with basis $\varnothing$ and dimension $0$). The partial derivatives are those of Directional derivatives and partial derivatives of a map $U\subseteq\mathbb{R}^m\to\mathbb{R}^n$.

If $f:U\to\mathbb R^n$ is totally differentiable at $a$, then $D_vf(a)$ exists for every $v\in\mathbb R^m$ and equals $Df(a)v$. In particular $\partial_jf(a)=Df(a)e_j$, and the matrix of $Df(a)$ is $Jf(a)$.

If a scalar-valued $f:U\to\mathbb R$ is totally differentiable at $a$, then $D_vf(a)=\langle\nabla f(a),v\rangle$ for every $v$. Among unit vectors $v$, this is at most $\|\nabla f(a)\|_2$; if the gradient is nonzero, equality holds exactly in the direction $\nabla f(a)/\|\nabla f(a)\|_2$. If the gradient is zero, every unit direction has directional derivative zero.

If $f,g:U\to\mathbb R^n$ are totally differentiable at $a$ and $c\in\mathbb R$, then $f+g$ and $cf$ are totally differentiable at $a$, with

$D(f+g)(a)=Df(a)+Dg(a),\qquad D(cf)(a)=cDf(a).$

Let $f:U\to V\subseteq\mathbb R^n$ be totally differentiable at $a\in U$ and let $g:V\to\mathbb R^p$ be totally differentiable at $f(a)$. Then $g\circ f$ is totally differentiable at $a$ and

$D(g\circ f)(a)=Dg(f(a))\circ Df(a).$

Let $B(a,r)\subseteq U\subseteq\mathbb R^m$ and let $\|h\|_2<r$. Define $p_0=a$ and $p_j=a+\sum_{i<j}h_ie_i$ for $1\le j\le m$. Then every $p_j$ lies in $B(a,r)$, $p_m=a+h$, and for every map $f$ on $U$,

$f(a+h)-f(a)=\sum_{j<m}\bigl(f(p_{j+1})-f(p_j)\bigr).$

Let $U\subseteq\mathbb R^m$ be open and let $f:U\to\mathbb R^n$. Suppose every partial derivative $\partial_jf$ exists on a neighbourhood of $a\in U$ and is continuous at $a$. Then $f$ is totally differentiable at $a$, and $Df(a)$ is the linear map with matrix $Jf(a)$.

Let $U\subseteq\mathbb R^m$ be convex and open, and let $f:U\to\mathbb R^n$ be totally differentiable at every point. If $M\ge0$ satisfies $\|Df(z)v\|_2\le M\|v\|_2$ for every $z\in U$ and $v\in\mathbb R^m$, then

$\|f(y)-f(x)\|_2\le M\|y-x\|_2\qquad(x,y\in U).$

Let $U\subseteq\mathbb R^m$ be convex and open. If $f:U\to\mathbb R^n$ is totally differentiable at every point and $Df(z)=0$ for every $z\in U$, then $f$ is constant on $U$.

The derivative definition is stated on open Euclidean domains so every sufficiently small increment is admissible. Its remainder uses the Euclidean norm; in finite-dimensional Euclidean spaces an equivalent norm would give the same differentiability notion, but that change is not part of this definition.

The linear-map definition on this page is deliberately the concrete Euclidean special case identified in Conventions of this page, the standing $n \ge 1$ hypothesis, and what is taken up elsewhere in the reading order. A future general linear-map development must prove agreement with A linear map $L:\mathbb{R}^m\to\mathbb{R}^n$ in Euclidean coordinates, not silently replace the meaning of $Df(a)$.