Exact stationary sensitivities and finite training derivatives are different

Let $p,m\ge 0$, let $\mathrm{\Theta }\subseteq {\mathbb{R}}^p$ be open with ${\theta }_0\in \mathrm{\Theta }$, let $D\subseteq {\mathbb{R}}^m$ be an open convex neighborhood of $z_{\ast }$, and let $G:U\to {\mathbb{R}}^m$ be $C^2$ on an open neighborhood $U$ of $D\times \{{\theta }_0\}$. Here $C^1$ means total differentiability with continuous derivative, $C^2$ means the derivative map is $C^1$, and vector and operator norms are Euclidean and induced Euclidean norms. Write $G_z:=D_{z}G$ and $G_{\theta }:=D_{\theta }G$ for the state and parameter derivatives. Suppose $z_{\ast }=G(z_{\ast },{\theta }_0)$, $G(D,{\theta }_0)\subseteq D$, $0\le q<1$, $\mathrm{\parallel }{G}_z(z,{\theta }_0)\mathrm{\parallel }\le q$ on $D$, and $G_z,G_{\theta }$ are Lipschitz in $z$ with constants $L_z,L_{\theta }$. Let $z_0:\mathrm{\Theta }\to {\mathbb{R}}^m$ be $C^1$, with all finite iterates at ${\theta }_0$ lying in $D$, and define $z_{k+1}(\theta )=G(z_k(\theta ),\theta )$. Let $\varphi$ be the exact local fixed-point branch, $s_{\ast }:=D\varphi ({\theta }_0)$, $e_K:=D{z}_K({\theta }_0)$, ${\delta }_0:=\mathrm{\parallel }{z}_0({\theta }_0)-z_{\ast }{\mathrm{\parallel }}_2$, $d_0:=\mathrm{\parallel }D{z}_0({\theta }_0)-s_{\ast }\mathrm{\parallel }$, and $C:=L_z\mathrm{\parallel }{s}_{\ast }\mathrm{\parallel }+L_{\theta }$. For every integer $K\ge 1$,

$\mathrm{\parallel }{e}_K-s_{\ast }\mathrm{\parallel }\le q^K{d}_0+CK{q}^{K-1}{\delta }_0,$

where $q^0=1$ when $q=0$ and $K=1$. For the fixed-point residual $r_K:=z_K({\theta }_0)-G(z_K({\theta }_0),{\theta }_0)$, the primal-state estimate is $(1-q)\mathrm{\parallel }{z}_K({\theta }_0)-z_{\ast }{\mathrm{\parallel }}_2\le \mathrm{\parallel }{r}_K{\mathrm{\parallel }}_2$. This primal residual estimate alone does not imply that the derivative of the finite $K$-step graph equals the exact fixed-point sensitivity.

For a separate finite shared-parameter recurrence, let $p_0,q_0\ge 1$, let ${\mathrm{\Theta }}_0\subseteq {\mathbb{R}}^{p_0}$ and $H\subseteq {\mathbb{R}}^{q_0}$ be open, let $T\ge 0$, let $h_0:{\mathrm{\Theta }}_0\to H$ and ${\mathrm{\Phi }}_t:{\mathrm{\Theta }}_0\times H\to H$ for $1\le t\le T$ and $\mathrm{\ell }:{\mathrm{\Theta }}_0\times H\to \mathbb{R}$ be $C^1$ on open domains containing the trajectory at $\theta \in {\mathrm{\Theta }}_0$, and suppose every piecewise primitive is evaluated in one unique strict open $C^1$ branch. Set $h_t={\mathrm{\Phi }}_t(\theta ,h_{t-1})$ and $J_T(\theta )=\mathrm{\ell }(\theta ,h_T)$; derivative matrices use Euclidean coordinates and transposes are Euclidean. Backpropagation differentiates this finite scalar graph: with $A_t=D_{\theta }{\mathrm{\Phi }}_t(\theta ,h_{t-1})$, $B_t=D_h{\mathrm{\Phi }}_t(\theta ,h_{t-1})$, ${\lambda }_T=(D_h\mathrm{\ell }(\theta ,h_T){)}^T$, and ${\lambda }_{t-1}=B_t^T{\lambda }_t$, its derivative is

${\mathrm{\nabla }}_{\theta }{J}_T(\theta )=(D_{\theta }\mathrm{\ell }(\theta ,h_T){)}^T+{\sum }_{t=1}^T{A}_t^T{\lambda }_t+(D{h}_0(\theta ){)}^T{\lambda }_0\mathrm{.}$

This is the derivative of the finite training computation. The exact implicit hypergradient is the derivative of a local stationary or fixed-point branch. In particular, a small training residual is not by itself an error bound for the derivative of a stopped iteration; contraction and sensitivity hypotheses such as those stated above are needed.

Finite fixed-point sensitivity and residual bounds

Gradient of a finite shared-parameter recurrence

Adjoint solve and measured residual

Exact local stationary-branch hypergradient