A constrained local margin minimum satisfies Clarke KKT under MFCQ

Write $f(\theta )=\frac{1}{2}\mathrm{\parallel }\theta {\mathrm{\parallel }}_2^2$ and $g_i(\theta )=1-q_i(\theta )$. Then $f$ is differentiable with ${\mathrm{\partial }}^{\circ }f({\theta }_{\ast })=\{{\theta }_{\ast }\}$, and ${\mathrm{\partial }}^{\circ }{g}_i({\theta }_{\ast })=-{\mathrm{\partial }}^{\circ }{q}_i({\theta }_{\ast })$ by the Clarke definition. Let $I$ be the finite active set $\{i:q_i({\theta }_{\ast })=1\}$.

Set $S:={\mathrm{\partial }}^{\circ }f({\theta }_{\ast })\cup {\bigcup }_{i\in I}{\mathrm{\partial }}^{\circ }{g}_i({\theta }_{\ast })$. Each constituent is compact and convex, so $S$ is bounded. In any finite convex combination of points of $S$, group together the points from each constituent; convexity then writes the combination using at most one point from each of the finitely many constituents. Therefore $C:=\mathrm{conv}(S)$ is the continuous image of the product of those constituents and their finite coefficient simplex. That product is closed and bounded in a finite-dimensional Euclidean space, hence compact by [F3], so $C$ is compact, nonempty, and convex.

If $0\mathrm{\notin }C$, [F4] strictly separates $0$ from $C$, giving a direction $v$ with $⟨h,v⟩<0$ for every $h\in S$. By [F1], this means $f^{\circ }({\theta }_{\ast };v)<0$ and $g_i^{\circ }({\theta }_{\ast };v)<0$ for every active $i$. The definition of each limsup gives a positive step bound for which each corresponding difference quotient along ${\theta }_{\ast }+tv$ is negative; taking the minimum of these finitely many bounds makes $f({\theta }_{\ast }+tv)<f({\theta }_{\ast })$ and $g_i({\theta }_{\ast }+tv)<0$ simultaneously for every active constraint. Continuity preserves the already strict inactive constraints. This is a feasible descent from a constrained local minimum, a contradiction. Hence $0\in C$.

Since $0\in \mathrm{conv}(S)$ and each Clarke subdifferential in $S$ is convex, group the finite convex combination by its constituent set. There are coefficients ${\alpha }_0\ge 0$ and ${\alpha }_i\ge 0$ for active $i$, with ${\alpha }_0+{\sum }_{i\in I}{\alpha }_i=1$, and vectors $h_0\in {\mathrm{\partial }}^{\circ }f({\theta }_{\ast })$, $h_i\in {\mathrm{\partial }}^{\circ }{g}_i({\theta }_{\ast })$ such that $0={\alpha }_0{h}_0+{\sum }_{i\in I}{\alpha }_i{h}_i$.

If there are no active constraints, the coefficients in step 4.1 sum to one, so ${\alpha }_0=1$. Otherwise, if ${\alpha }_0=0$, some active coefficient is positive; dotting the equality in step 4.1 with the MFCQ direction from [F2] makes the right-hand sum strictly negative, a contradiction. In either case ${\alpha }_0>0$. Divide by ${\alpha }_0$, set ${\lambda }_i={\alpha }_i\mathrm{/}{\alpha }_0$ for active indices and ${\lambda }_i=0$ otherwise, and put ${\tilde{h}}_i=-h_i\in {\mathrm{\partial }}^{\circ }{q}_i({\theta }_{\ast })$. Then ${\theta }_{\ast }={\sum }_i{\lambda }_i{\tilde{h}}_i$, all ${\lambda }_i\ge 0$, and ${\lambda }_i(q_i({\theta }_{\ast })-1)=0$ for every $i$. These are exactly the margin-form Clarke KKT conditions in [F2].