Bilevel learning, hypergradients, and implicit layers — Examples

Differentiate the inner loss: ${\mathrm{\partial }}_{w}f=(1+\lambda )w-1$ and $H=1+\lambda >0$. Thus its stationary point is $w^{\ast }=1\mathrm{/}(1+\lambda )$. Completing the square gives $f(\lambda ,w)=\frac{1}{2}(1+\lambda )(w-\frac{1}{1+\lambda }{)}^2+\frac{\lambda }{2(1+\lambda )}$, so this stationary point is the unique global minimizer.

Substitution into the outer loss gives $J(\lambda )=1\mathrm{/}(2(1+\lambda {)}^2)$, whose direct derivative is $J^{\mathrm{\prime }}(\lambda )=-1\mathrm{/}(1+\lambda {)}^3$.

At $w^{\ast }$, the mixed derivative is $B={\mathrm{\partial }}_{\lambda }((1+\lambda )w-1)=w^{\ast }=1\mathrm{/}(1+\lambda )$, the state derivative is ${\mathrm{\nabla }}_{w}g=w^{\ast }=1\mathrm{/}(1+\lambda )$, and ${\mathrm{\partial }}_{\lambda }g=0$. Formula [F2] therefore gives $-B{H}^{-1}{\mathrm{\nabla }}_{w}g=-1\mathrm{/}(1+\lambda {)}^3$, agreeing with step 2.1.

The inner loss is nonnegative and equals zero exactly when $u=\theta$, independently of $v$. Therefore its full global minimizer set is $\{(\theta ,v):v\in \mathbb{R}\}$.

Its state gradient is $(u-\theta ,0{)}^T$ and its Hessian is $\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)$, whose kernel contains $(0,1{)}^T$ and is therefore singular.

The maps $w_0^{\ast }(\theta )=(\theta ,0)$ and $w_1^{\ast }(\theta )=(\theta ,\theta )$ are both $C^1$ and lie in the minimizing set from step 1.1 for every $\theta$. Their outer objectives are $g(w_0^{\ast }(\theta ),\theta )=0$ and $g(w_1^{\ast }(\theta ),\theta )=\theta$, so their derivatives are respectively $0$ and $1$.

Since these two admissible selections yield distinct derivatives and step 1.2 violates the invertibility hypothesis in [F2], the input does not determine one hypergradient through the argmin set, and the local formula does not apply.

Solving $z=G(z,\theta )$ gives $(1-q)z=(1-q)\theta$. Since $1-q>0$, the unique fixed point is $z^{\ast }(\theta )=\theta$, and therefore $J(\theta )=z^{\ast }(\theta )$ has derivative $1$. Also $G_z=q<1$.

For $K=0$, $z_0(\theta )=(1-q^0)\theta =0$. If $z_K(\theta )=(1-q^K)\theta$, then $z_{K+1}=q(1-q^K)\theta +(1-q)\theta =(1-q^{K+1})\theta$. Induction gives the displayed finite-iterate formula for every $K\ge 1$.

The inner objective has ${\mathrm{\partial }}_{z}f=z-\theta$, so its stationary branch is the same $z^{\ast }(\theta )=\theta$ as the fixed-point branch. Here $H=1$, $B=-1$, ${\mathrm{\partial }}_{\theta }g=0$, and ${\mathrm{\nabla }}_{z}g=1$. Formula [F3] therefore gives $0-(-1)(1{)}^{-1}(1)=1$, verifying the exact hypergradient under the bilevel prerequisites.

At ${\theta }_0=0$, step 1.2 gives $z_K(0)=0=z^{\ast }(0)$ and the fixed-point residual is $z_K(0)-G(z_K(0),0)=0-G(0,0)=0$. The stopped outer objective is $J_K(\theta )=g(z_K(\theta ),\theta )=(1-q^K)\theta$, so $J_K^{\mathrm{\prime }}(0)=1-q^K$.

Because $0<q<1$ and $K\ge 1$, $q^K>0$, hence $1-q^K\mathrm{\ne }1$. Thus the zero residual at the exact state does not make the finite-unroll derivative equal to the exact hypergradient.