The signed-square ReLU differential need not be locally Lipschitz

On a small neighborhood of $(1,1)$ with $w,b>0$, the root $c(w,b)$ is a smooth function of parameters. For an increment $h=(h_1,h_2)$, $r_{(w,b)+h}(x)-r_{w,b}(x)=2w{h}_{1}x+2b{h}_2+h_1^{2}x+h_2^2\mathrm{.}$ The candidate differential is $D{\mathrm{\Phi }}_{(w,b)}h={\mathbf{1}}_{x>c(w,b)}(2w{h}_{1}x+2b{h}_2)$. Outside the interval between $c(w,b)$ and $c((w,b)+h)$, the two preactivations have the same sign; there the ReLU remainder is either zero or $h_1^{2}x+h_2^2$, whose $L^2(pdx)$ norm is $O(\mathrm{\mid }h{\mathrm{\mid }}^2)$ by the finite second moment in [F1]. The interval between the two roots has length $O(\mathrm{\mid }h\mathrm{\mid })$ and lies in a fixed bounded neighborhood of $-1$; on it the pointwise remainder is $O(\mathrm{\mid }h\mathrm{\mid })$, so its $L^2(pdx)$ norm is $O(\mathrm{\mid }h{\mathrm{\mid }}^{3\mathrm{/}2})$. Thus the full remainder is $o(\mathrm{\mid }h\mathrm{\mid })$, proving Fréchet differentiability and the displayed differential.

At $(1,1)$ the derivative in direction $e_w=(1,0)$ is $2x{\mathbf{1}}_{x>-1}$. At $(1,1+\delta )$ it is $2x{\mathbf{1}}_{x>-(1+\delta {)}^2}$. For $0<\delta \le 1\mathrm{/}4$, their squared $L^2$ distance is $4{\int }_{-(1+\delta {)}^2}^{-1}{x}^{2}p(x)dx\ge 8p(2)\delta ,$ because the interval lies in $[-2,-1]$, where $x^2\ge 1$ and $p(x)\ge p(2)$, and has length $2\delta +{\delta }^2\ge 2\delta$. The operator norm difference of the two differentials is at least their difference on the unit vector $e_w$. Dividing by the parameter distance $\delta$ gives a quotient at least $\sqrt{8p(2)}\mathrm{/}\sqrt{\delta }$, which diverges as $\delta \downarrow 0$. Therefore the differential is not locally Lipschitz at $(1,1)$.