Mixed Partials, Taylor Formulae, and Extrema

Apply [L1] in the $x$ variable to $x\mapsto f(x,y_1)-f(x,y_0)$. The stated existence of $f_x$ on an open neighbourhood gives the required one-variable regularity, and the rectangle difference is $(x_1-x_0)(f_x(\xi ,y_1)-f_x(\xi ,y_0))$.

Apply [L1] in the $y$ variable to $y\mapsto f_x(\xi ,y)$. Since $f_{xy}$ exists on an open neighbourhood, this one-variable map is continuous on the closed interval and differentiable on its interior. This yields $(y_1-y_0)f_{xy}(\xi ,\eta )$ and proves the formula.

For sufficiently small nonzero $h,k$, apply [L1] to the rectangle with corners $(a,b)$ and $(a+h,b+k)$. After division by $hk$, continuity of $f_{xy}$ at $(a,b)$ makes the limit, as $h,k\to 0$, equal to $f_{xy}(a,b)$.

For fixed nonzero $h$, first let $k\to 0$ in the same rectangle quotient; it becomes $(f_y(a+h,b)-f_y(a,b))\mathrm{/}h$. Letting $h\to 0$ gives the defining quotient for $f_{yx}(a,b)$.

The two limits are equal, proving $f_{xy}(a,b)=f_{yx}(a,b)$.

By [L1] and [L2], write $D{f}_x(a,b)(s,t)=As+Bt$ and $D{f}_y(a,b)(s,t)=Cs+Dt$.

By [L3], for small nonzero $h$, define the following rectangular difference.

Apply [L3] instead to $y\mapsto f(a+h,y)-f(a,y)$ on the interval with endpoints $b,b+h$.

Steps 2.1 and 2.2 give $(B-C)h^2=o(h^2)$. Divide by $h^2$ and let $h\to 0$ to obtain $B=C$, hence $f_{xy}(a,b)=f_{yx}(a,b)$.

At an arbitrary point, [L2] supplies ${\mathrm{\partial }}_i{\mathrm{\partial }}_{j}f$ on a neighbourhood and continuously there, as well as the reversed partial ${\mathrm{\partial }}_j{\mathrm{\partial }}_{i}f$ at the point.

Apply [L1] at an arbitrary point to obtain ${\mathrm{\partial }}_i{\mathrm{\partial }}_{j}f={\mathrm{\partial }}_j{\mathrm{\partial }}_{i}f$.

For every $i,j$, [L1] and [L2] give $(H_f(a){)}_{ij}={\mathrm{\partial }}_j{\mathrm{\partial }}_{i}f(a)=(H_f(a){)}_{ji}$.

Entrywise equality with the transpose proves symmetry.

Label the $k$ differentiation positions and count inversions of a permutation of these labels. A permutation with zero inversions is the identity, so it leaves the derivative unchanged.

Assume every reordering with at most $n$ inversions leaves the derivative unchanged.

A reordering with $n+1$ inversions has an adjacent inverted pair; exchanging that pair reduces its inversion count by one. If that pair occupies positions $r,r+1$ in the sequence of differentiation operations, first apply only the operations in positions $1,\dots ,r-1$ and call the resulting field $g$. Every ordered partial of $g$ through order two is an ordered partial of $f$ of length at most $k$, hence is continuous by [L2]; thus $g\in C^2(U)$ and [L1] swaps precisely the operations in positions $r,r+1$. Apply the remaining outer operations in positions $r+2,\dots ,k$ to this equality; their existence is again supplied by [L2].

The induction hypothesis applies after the swap in step 2.1, so the original reordering leaves the derivative unchanged. Induction on inversion number proves the claim for every finite permutation.

For $r=0$ the displayed sum consists of the zero multi-index and equals $f(a+th)=g(t)$.

Fix $r<k$ and assume the formula at order $r$.

Each $D^{\alpha }f$ with $\mathrm{\mid }\alpha \mathrm{\mid }=r$ has continuous first partials, so [L1] differentiates its composition with the affine line. By [L3], we use the canonical multi-index notation for the resulting derivatives. When $r=0$ the resulting first derivatives are already canonical; when $r\ge 1$, [L2] permits the resulting derivatives to be written as $D^{\alpha +e_i}f$. By [L4], collecting the coefficient of a fixed $\beta$ with $\mathrm{\mid }\beta \mathrm{\mid }=r+1$ gives

Steps 1.1--2.1 prove the formula successively for every $r\le k$.

Put $I:=\{t\in \mathbb{R}:a+th\in U\}$ and $g(t):=f(a+th)$. The set $I$ is open, contains $[0,1]$ by [L1], and is an interval because $U$ is convex. Hence [L2] shows that $g$ has the derivatives through order $k+1$ required by [L3].

Apply [L3] to $g$ between $0$ and $1$.

Substitute the formula of [L2] for $g^{(j)}(0)$ and $g^{(k+1)}(\theta )$; the degree-$k$ part is the definition of $T_{k}f(a;h)$.

Subtract the degree-$k$ Taylor polynomial from the equality of [L1]. The remainder is the following.

For $h\mathrm{\ne }0$, divide the absolute value in step 1.1 by $\mathrm{\parallel }h{\mathrm{\parallel }}^k$. Since $\mathrm{\mid }{h}^{\alpha }\mathrm{\mid }\le \mathrm{\parallel }h{\mathrm{\parallel }}^k$, it is bounded by the finite sum of the coefficient differences divided by $\iota (\alpha !)$. As $h\to 0$, also $a+{\theta }_{h}h\to a$, so every term tends to zero by [L2].

This proves that the remainder in step 1.1 is $o(\mathrm{\parallel }h{\mathrm{\parallel }}^k)$, and the subtracted polynomial is $T_{k}f(a;h)$ by The multivariable Taylor polynomial in multi-index notation.

Expand the degree-one and degree-two multi-index sums in [L1].

The degree-one sum is $\mathrm{\nabla }f(a)\cdot h$, while symmetry of the repeated second derivatives identifies the degree-two sum with $\frac{1}{2}⟨H_f(a)h,h⟩$.

Restrict $f$ to each coordinate line through $a$. The restriction has a local extremum at $0$, so [L1] makes its derivative zero.

These derivatives are the entries of $\mathrm{\nabla }f(a)$ by [L2], so every entry vanishes.

The unit sphere is nonempty because it contains $e_0$, closed by [L1], and bounded since it lies in the radius-two ball about zero.

It is compact by [L2].

The finite coordinate formula for $q$ makes it continuous; [L3] therefore gives a point where $q$ attains its minimum and maximum on the sphere.

In the positive definite case the attained minimum is positive, and in the negative definite case the attained maximum is negative, by the definition of definiteness.

Taking $c$ to be the positive minimum or the negative maximum gives the asserted uniform signed bounds.

At a critical point the linear term in [L1] vanishes.

Write $h=ru$ with $r=\mathrm{\parallel }h{\mathrm{\parallel }}_2$ and $\mathrm{\parallel }u{\mathrm{\parallel }}_2=1$ when $h\mathrm{\ne }0$. The sign bound in [L2] dominates the $o(r^2)$ remainder for sufficiently small $r$.

This gives the strict minimum and maximum conclusions in the definite cases; two unit directions of opposite quadratic sign give neither extremum in the indefinite case.

For a semidefinite Hessian which is not definite, its quadratic form has a nonzero null direction, so step 2.1 supplies no signed quadratic bound. No universal conclusion is possible: at $0$ the one-variable functions $x^4$, $-x^4$, and $x^3$ all have zero Hessian, but respectively have a strict local minimum, a strict local maximum, and neither.

By [L1], the Hessian quadratic form is $Q(s,t)=A{s}^2+2Bst+C{t}^2$. If $A\mathrm{\ne }0$, completing the square gives $Q=A(s+(B\mathrm{/}A)t{)}^2+(\mathrm{\Delta }\mathrm{/}A)t^2$.

If $\mathrm{\Delta }>0$, the two coefficients in step 1.1 have the sign of $A$, so $Q$ is positive definite for $A>0$ and negative definite for $A<0$.

If $\mathrm{\Delta }<0$ and $A\mathrm{\ne }0$, then $Q(1,0)=A$ while $Q(-B\mathrm{/}A,1)=\mathrm{\Delta }\mathrm{/}A$, which have opposite signs; hence $Q$ is indefinite. If $A=0$, then $-B^2=\mathrm{\Delta }<0$, so $B\mathrm{\ne }0$, and $Q(1,t)=2Bt+C{t}^2$ has both signs for sufficiently small positive and negative $t$.

Apply [L2] to steps 2.1 and 2.2. When $\mathrm{\Delta }=0$ and $A\mathrm{\ne }0$, step 1.1 makes $Q=A(s+(B\mathrm{/}A)t{)}^2$, so it is semidefinite but not definite; when $A=0$, then $B=0$ and $Q=C{t}^2$, with the same conclusion (including $Q=0$). Thus this is the inconclusive case of [L2].

The composite $g=f\circ \gamma$ is differentiable at $0$ by [L1], and $0$ is an interior local extremum of $g$ by the hypothesis.

Fermat's theorem gives $g^{\mathrm{\prime }}(0)=0$.

The chain-rule identity in [L1] and $\gamma (0)=a$ give $g^{\mathrm{\prime }}(0)=Df(a){\gamma }^{\mathrm{\prime }}(0)$. Combining with step 2.1 proves the conclusion.

Parametrize the graph by $\mathrm{\Gamma }(x)=(x,\psi (x))$. Since $\psi$ is differentiable at $x_0$, $\mathrm{\Gamma }$ is continuous there; because $V$ and $W$ are open, for every $v\in {\mathbb{R}}^m$ the curve $t\mapsto \mathrm{\Gamma }(x_0+tv)$ is defined and lies in $W$ for sufficiently small $t$. Apply [L1] to get $Df(a)(v,D\psi (x_0)v)=0$.

In block gradient coordinates, step 1.1 says ${\mathrm{\nabla }}_{x}f(a)+D\psi (x_0{)}^T{\mathrm{\nabla }}_{y}f(a)=0$.

Set $\lambda ={\mathrm{\nabla }}_{y}f(a)$. Since $DG(a)=(-D\psi (x_0),I_n)$, step 2.1 yields $DG(a{)}^T\lambda =({\mathrm{\nabla }}_{x}f(a),{\mathrm{\nabla }}_{y}f(a))=\mathrm{\nabla }f(a)$.