Real-arithmetic graph derivatives and rounded execution are different objects

The exact AD identities refer to the real-arithmetic graph and its exact primitive derivatives. A floating execution uses rounded primal values and derivative arithmetic; its returned AD quantity is not asserted to be the ordinary derivative of the end-to-end rounded machine map. The theorem Local Jacobian perturbations give a pathwise bound for graph-gradient error propagates specified local Jacobian discrepancies with exact accumulation and excludes rounding in reverse accumulation additions.

For a concrete mathematical solve primitive, Certified inverse defects and primal residuals bound linear-solve Jacobian and pullback errors supplies those discrepancies. With stored primal $\widehat{x}$, fixed approximate inverse $B$ assembled from basis solves, certified $\mathrm{\parallel }I-AB{\mathrm{\parallel }}_2\le d<1$, $\mathrm{\parallel }b-A\widehat{x}{\mathrm{\parallel }}_2\le \rho$, $\mathrm{\parallel }B{\mathrm{\parallel }}_2\le M$, and $K=M\mathrm{/}(1-d)$, the vector and matrix edges have ${\eta }_b=Kd$ and ${\eta }_A=Kd\mathrm{\parallel }\widehat{x}\mathrm{\parallel }+K^2\rho$. Its additional explicit dot/outer-product allowances cover rounded pullback evaluation. Triangular, successful Cholesky, and QR backends are admissible once those residuals are certified; a small factorization residual alone is insufficient. The lemma includes the derivative-evaluation-point error, conditioning, arithmetic success and replay hypotheses.

For the implicit example, specify $A(\theta )=\left[\begin{array}{cc}1& \theta \\ 0& 1\end{array}\right]$, $b=(2,1{)}^T$, $x(\theta )=A(\theta {)}^{-1}b$, and objective $J(\theta )=(1,1)x(\theta )$. Then $x(\theta )=(2-\theta ,1{)}^T$ and $J^{\mathrm{\prime }}(0)=-1$. At $\theta =0$, ordinary triangular substitution for the primal and basis solves gives $\widehat{x}=(2,1{)}^T$, $B=I$, and $d=\rho =0$. In Operationwise real and complex floating point arithmetic with rounded square roots and fused multiply-add, assume $1,2,3$ are normal representable numbers. The solves, sequential dot products for $B^{T}c$ with $c=(1,1{)}^T$, and outer product $-(B^{T}c){\widehat{x}}^T$ have only representable results $0,\pm 1,\pm 2$. Round-to-nearest therefore leaves every result exact. Contracting the matrix cotangent $-\left[\begin{array}{cc}2& 1\\ 2& 1\end{array}\right]$ with $A^{\mathrm{\prime }}(0)=\left[\begin{array}{cc}0& 1\\ 0& 0\end{array}\right]$ returns $-1$. This exactness follows from the specified operations and rounding rule; zero solve defects alone only remove the fixed-map discrepancy.

A Cholesky or QR factor-output primitive is a different smooth map from the mathematical solve. Its derivative must be evaluated on one fixed strict branch, with its own local derivative errors, or its finite loop must be expanded into the scalar primitives of the graph. The solve bound does not identify a factorization residual with a factor-output derivative bound.