LemmaStatement: Literature-sourcedProof: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Gradient descent induces an exact output Gram update with a Taylor remainder

Statement

For any fixed finite real network obeying either architecture recurrence defined on the A page, let $u (θ) = (f (x_{i}, θ))_{i = 1}^{n} \in R^{n}$ , let $y = (y_{i})_{i = 1}^{n}$ , let $J (θ) \in R^{n \times p}$ have row $i$ equal to the gradient of $f (x_{i}, θ)$ with respect to the vector of all trainable parameters, and set $G (θ) = J (θ) J (θ)^{T}$ . For the empirical quadratic loss $L (θ) = \frac{1}{2} ∥ u (θ) - y ∥_{2}^{2}$ , one full-batch gradient-descent step with step size $η > 0$ satisfies

$u (θ^{+}) = u (θ) - η G (θ) (u (θ) - y) + R,$

where $θ^{+} = θ - η J (θ)^{T} (u (θ) - y)$ and $R$ is the first-order Taylor remainder of the differentiable map $u$ at $θ$ in direction $- η J (θ)^{T} (u (θ) - y)$ . No size bound on $R$ is asserted here.

Statement references

Du et al., gradient-descent dynamics, §§4 and B.1

Facts & Assumptions

Given: One finite network, its trainable parameter vector, a fixed data set and labels, and a step size $η > 0$ .

[F1]

The network is a finite composition of linear maps and an analytic coordinatewise activation, so its output map is continuously differentiable in all trainable parameters. Fully connected and residual networks with the Du Gram kernel

[F2]

Reverse accumulation through a finite $C^{1}$ parameterized recurrence gives the exact parameter gradient, including all parameter uses. Reverse accumulation sums gradients over shared parameter uses

[F3]

Fixed-step gradient descent updates the parameter by subtracting $η$ times the Euclidean gradient. Gradient descent

[F4]

Differentiability in finite-dimensional real parameter spaces means a linear first-order approximation with a remainder $o (∥ h ∥_{2})$ . The real Frechet derivative on real and complex matrix spaces with the Frobenius norm

Proof

technique · express the loss gradient by the output Jacobian and apply the defining first-order expansion of the output map

1.1F2F3givenalgebra

Write $r = u (θ) - y$ . By the scalar chain rule applied to $\frac{1}{2} \sum_{i} (u_{i} - y_{i})^{2}$ , the parameter gradient is $J (θ)^{T} r$ . By [F3], $h : = θ^{+} - θ = - η J (θ)^{T} r$ .

2.1F1F4step 1.1algebra∎

Since $u$ is differentiable, [F4] gives $u (θ + h) = u (θ) + J (θ) h + R$ , where $R : = u (θ + h) - u (θ) - J (θ) h$ is the first-order remainder. Substitute $h = - η J (θ)^{T} r$ to get $J (θ) h = - η J (θ) J (θ)^{T} r = - η G (θ) r$ , which is the claimed identity.

Depends on

Used by

Dependency tree · two levels

17 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Simon S. Du, Jason D. Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai, Gradient Descent Finds Global Minima of Deep Neural Networks, §§4 and B.1 (standard reference, not scraped)