Clarke subdifferential and gradient-flow trajectory

Let $d\ge 1$, let $U\subseteq {\mathbb{R}}^d$ be open, and let $F:U\to \mathbb{R}$ be locally Lipschitz. For $x\in U$ and $v\in {\mathbb{R}}^d$, define the Clarke generalized directional derivative by

$F^{\circ }(x;v):={\mathrm{lim\hspace{0.17em}sup}}_{z\to x,s\downarrow 0,z,z+sv\in U}\frac{F(z+sv)-F(z)}{s}\mathrm{.}$

The Clarke subdifferential (also called the Clarke generalized gradient) is

${\mathrm{\partial }}^{\circ }F(x):=\{\xi \in {\mathbb{R}}^d:⟨\xi ,v⟩\le F^{\circ }(x;v)\text{ for every }v\in {\mathbb{R}}^d\}\mathrm{.}$

For locally Lipschitz $F$, this is a nonempty compact convex set and is equivalently the closed convex hull of all limits of ordinary gradients $\mathrm{\nabla }F(x_k)$ along differentiability points $x_k\to x$. Its support function is $F^{\circ }(x;v)={\max }_{\xi \in {\mathrm{\partial }}^{\circ }F(x)}⟨\xi ,v⟩$. The pairing is Euclidean: $⟨a,b⟩={\sum }_{j=1}^d{a}_j{b}_j$.

The set-valued map $x\mapsto {\mathrm{\partial }}^{\circ }F(x)$ is locally bounded and has closed graph: if $x_k\to x$, ${\xi }_k\in {\mathrm{\partial }}^{\circ }F(x_k)$, and ${\xi }_k\to \xi$, then $\xi \in {\mathrm{\partial }}^{\circ }F(x)$. Here locally bounded means that every point has a neighborhood on which all subgradients have bounded norm. For each fixed $v$, the function $x\mapsto F^{\circ }(x;v)$ is upper semicontinuous: if $a>F^{\circ }(x;v)$, some neighborhood $V$ of $x$ satisfies $F^{\circ }(y;v)<a$ for all $y\in V$.

A locally Lipschitz function $F$ admits the chain rule along absolutely continuous arcs if, for every arc $z:[0,T]\to U$ that is absolutely continuous on the compact interval $[0,T]$, for almost every $t\in [0,T]$ the derivative $(F\circ z{)}^{\mathrm{\prime }}(t)$ exists and equals $⟨\xi ,\dot{z}(t)⟩$ for every $\xi \in {\mathrm{\partial }}^{\circ }F(z(t))$.

A global Clarke gradient-flow trajectory for $F$ is a curve $\theta :[0,\mathrm{\infty })\to U$ that is absolutely continuous on every compact time interval and satisfies $\dot{\theta }(t)\in -{\mathrm{\partial }}^{\circ }F(\theta (t))$ for almost every $t\ge 0$. The definition names a trajectory when it exists; it does not assert existence or continuation.

Kaifeng Lyu and Jian Li, Gradient Descent Maximizes the Margin of Homogeneous Neural Networks, Appendix I.2–I.3

Matus Telgarsky, Deep Learning Theory lecture notes, §9.1