Local C1 step consistency gives tangent, adjoint and gradient order

The exact objective is also the finite graph with maps ${\mathrm{\Phi }}_n$, costs $K_n$, initial state $u(0)$ and terminal cost $\varphi$: the intervals partition the integral. Differentiability of $K_n$ follows by differentiating its integral using the common-tube flow derivative and the uniform continuous derivative bounds. In particular the initial-state variational matrix $V$ on interval $n$ has $V(t_n)=I$, $V^{\mathrm{\prime }}=f_{u}V$, and $c_n^e={\int }_{t_n}^{t_{n+1}}V(t{)}^T{\mathrm{\ell }}_u(t{)}^{T}dt\mathrm{.}$ The pairing identity gives $p(t_n)=c_n^e(u(t_n),\theta )+A_n^e(u(t_n),\theta {)}^{T}p(t_{n+1})\mathrm{.}$ Thus the exact-flow graph adjoint equals the continuous adjoint at every node. Its tangent and gradient are also those of the exact continuous objective.

Write $u_n=u(t_n)$ and $e_n=\mathrm{\parallel }{x}_n-u_n\mathrm{\parallel }$. For $d\ge 1$, parameterize the admitted segment by $s\mapsto u_n+s(x_n-u_n)$, $0\le s\le 1$. The chain rule and the derivative bound for ${\mathrm{\Psi }}_n$ bound the derivative of its composition with this segment by $(1+Lh)\mathrm{\parallel }{x}_n-u_n\mathrm{\parallel }$; the vector mean value inequality [F1] therefore gives its endpoint Lipschitz bound with constant $1+Lh$. For $d=0$, the state difference and both endpoint values lie in the zero vector space, so the same bound is immediate. Consequently $e_{n+1}\le (1+Lh)e_n+c_v{h}^{p+1}\mathrm{.}$ By [F3], $e_n\le X{h}^p$.

The exact tangent and adjoint recurrences give $\mathrm{\parallel }{S}_{n+1}\mathrm{\parallel }\le (1+Lh)\mathrm{\parallel }{S}_n\mathrm{\parallel }+Mh,\mathrm{\parallel }p(t_n)\mathrm{\parallel }\le (1+Lh)\mathrm{\parallel }p(t_{n+1})\mathrm{\parallel }+Mh\mathrm{.}$ Expanding these finite recurrences and using $(1+Lh{)}^k\le e^{LT}$ shows $\mathrm{\parallel }{S}_n\mathrm{\parallel }\le \bar{S}$ and $\mathrm{\parallel }p(t_n)\mathrm{\parallel }\le \bar{P}$. These bounds include $L=0$. For any one of the four derivative maps, comparison at numerical and exact nodes gives $\mathrm{\parallel }{F}_n^h(x_n,\theta )-F_n^e(u_n,\theta )\mathrm{\parallel }\le c_1{h}^{p+1}+Hh{e}_n\le R{h}^{p+1}\mathrm{.}$

The actual differentiated numerical graph satisfies $S_{n+1}^h=A_n^h(x_n)S_n^h+B_n^h(x_n)$. Subtract the exact recurrence and split the matrix product using $A_n^h(S_n^h-S_n)+(A_n^h-A_n^e)S_n$. Hence $\mathrm{\parallel }{S}_{n+1}^h-S_{n+1}\mathrm{\parallel }\le (1+Lh)\mathrm{\parallel }{S}_n^h-S_n\mathrm{\parallel }+R(\bar{S}+1)h^{p+1}\mathrm{.}$ The initial derivative bound and [F3] yield $C_S{h}^p$.

At the terminal node, the difference of adjoint seeds is at most $D{h}^p$. Subtract the two backward recurrences, splitting $A_h^T{p}_h-A_e^T{p}_e=A_h^T(p_h-p_e)+(A_h-A_e{)}^T{p}_e$. The Euclidean operator norm of a transpose equals that of the matrix. For every finite-dimensional Euclidean vector $v$, Cauchy–Schwarz gives $\mathrm{\parallel }v\mathrm{\parallel }={\sup }_{\mathrm{\parallel }w\mathrm{\parallel }\le 1}\mathrm{\mid }{w}^{T}v\mathrm{\mid }$: if $v\mathrm{\ne }0$, equality is attained at $w=v\mathrm{/}\mathrm{\parallel }v\mathrm{\parallel }$, while for $v=0$ both sides vanish, including in dimension zero. Applying this closed-unit-ball formula to both sides of the pairing gives the transpose identity in all finite dimensions. Therefore $\mathrm{\parallel }{p}_n^h-p(t_n)\mathrm{\parallel }\le (1+Lh)\mathrm{\parallel }{p}_{n+1}^h-p(t_{n+1})\mathrm{\parallel }+R(\bar{P}+1)h^{p+1}\mathrm{.}$ Reverse the indices and apply [F3]. This gives $C_P{h}^p$.

Apply the finite-graph gradient formula to both graphs: $\mathrm{\nabla }{J}_h=D_{\theta }{\varphi }_h(x_N{)}^T+(S_0^h{)}^T{p}_0^h+{\sum }_{n<N}\{d_n^h(x_n)+(B_n^h(x_n){)}^T{p}_{n+1}^h\},$ $\mathrm{\nabla }J=D_{\theta }\varphi (u_N{)}^T+S_0^{T}p(0)+{\sum }_{n<N}\{d_n^e(u_n)+(B_n^e(u_n){)}^{T}p(t_{n+1})\}\mathrm{.}$ The terminal direct-derivative difference is at most $D{h}^p$. The initial contribution is bounded by $\{c_{s0}\bar{P}+(M_0+c_{s0})C_P\}{h}^p,$ since $h\le 1$ implies $\mathrm{\parallel }{S}_0^h\mathrm{\parallel }\le M_0+c_{s0}$. Each running difference is bounded by $\{R+R\bar{P}+M{C}_P\}{h}^{p+1},$ using $B_h^T(p_h-p)+(B_h-B_e{)}^{T}p$. Summing $N$ terms and using $Nh=T$ proves the stated $C_G$ bound.