Clarke chain rule for finite exponential empirical risk

Put $f_i(\theta )=e^{-q_i(\theta )}$. At every differentiability point of $q_i$, ordinary differentiation gives $\mathrm{\nabla }{f}_i=-e^{-q_i}\mathrm{\nabla }{q}_i$. The exponential has nonzero derivative, so $f_i$ is differentiable exactly where $q_i$ is; passing to limits of gradients and then closed convex hulls as in [F1] gives ${\mathrm{\partial }}^{\circ }{f}_i(\theta )=-e^{-q_i(\theta )}{\mathrm{\partial }}^{\circ }{q}_i(\theta )$. Along an absolutely continuous arc, the assumed chain rule for $q_i$ and the one-variable chain rule give $(f_i\circ \theta {)}^{\mathrm{\prime }}=-e^{-q_i}⟨h_i,\dot{\theta }⟩$ for every $h_i\in {\mathrm{\partial }}^{\circ }{q}_i(\theta )$.

For any direction $v$, the limsup of the finite sum of difference quotients is at most the sum of their limsups, so ${\mathcal{L}}^{\circ }(\theta ;v)\le {\sum }_i{f}_i^{\circ }(\theta ;v)$. The right side is the support function of the nonempty compact convex set $C:={\sum }_i{\mathrm{\partial }}^{\circ }{f}_i(\theta )$ by [F1]. If some $g\in {\mathrm{\partial }}^{\circ }\mathcal{L}(\theta )$ lay outside $C$, [F3] would give a direction $v$ with $⟨g,v⟩>{\sup }_{c\in C}⟨c,v⟩={\sum }_i{f}_i^{\circ }(\theta ;v)$, contradicting $⟨g,v⟩\le {\mathcal{L}}^{\circ }(\theta ;v)$. Thus ${\mathrm{\partial }}^{\circ }\mathcal{L}(\theta )\subseteq C$. Using step 1.1 expands this inclusion to the displayed weighted-margin form.

Summing the ordinary arc derivatives from step 1.1 gives the displayed derivative formula. For a negative Clarke gradient-flow arc, step 2.1 supplies $h_i$ with $\dot{\theta }={\sum }_i{e}^{-q_i}{h}_i$; substitution makes the derivative $-⟨\dot{\theta },\dot{\theta }⟩=-\mathrm{\parallel }\dot{\theta }{\mathrm{\parallel }}_2^2$.