TheoremStatement: Literature-sourcedProof: AI-adaptedPipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

The nonlinear least-squares gradient and Hessian formulas

Statement

Let $n, m \geq 1$ , let $U \subseteq R^{n}$ be open, and let $r = (r_{1}, \dots, r_{m})^{T} : U \to R^{m}$ be of class $C^{2}$ (each component has continuous partial derivatives through order two, as in $C^{k}$ maps and multi-index derivative notation in Euclidean space). Define $ϕ (x) : = \frac{1}{2} \sum_{i = 1}^{m} r_{i} (x)^{2} = \frac{1}{2} ∥ r (x) ∥_{2}^{2},$ and let $J (x) \in R^{m \times n}$ be the Jacobian with $i$ th row $\nabla r_{i} (x)^{T}$ . Then $ϕ$ is of class $C^{2}$ and $\nabla ϕ (x) = J (x)^{T} r (x), \nabla^{2} ϕ (x) = J (x)^{T} J (x) + \sum_{i = 1}^{m} r_{i} (x) \nabla^{2} r_{i} (x) .$

The second term in the Hessian is the residual-curvature term. It vanishes when $r (x) = 0$ , but need not vanish at a nonzero-residual stationary point.

Facts & Assumptions

Given: The open set $U$ , the $C^{2}$ residual components $r_{i}$ , and the definitions of $ϕ$ , $J$ , and the Hessian in the Statement.

[F1]

$C^{k}$ maps and multi-index derivative notation in Euclidean space: class $C^{2}$ means each ordered iterated coordinate partial through order two exists and is continuous.

[F2]

The Jacobian matrix of partial derivatives and the gradient in the scalar-valued case: $J_{i j} = \partial_{j} r_{i}$ , and the gradient of a scalar field is the vector of its first partial derivatives.

[F3]

The Hessian matrix and critical points of a scalar field: the Hessian entries are $\partial_{k} \partial_{j} ϕ$ in the standard coordinates.

[F4]

If all partial derivatives exist on a neighbourhood and are continuous at a point, then the map is totally differentiable there with Jacobian derivative: if the partial derivatives of a map exist on a neighborhood and are continuous at a point, the map is totally differentiable there with its Jacobian as derivative.

[F5]

The chain rule for total derivatives: $D (g \circ f) (a) = D g (f (a)) \circ D f (a)$ : the derivative of a differentiable composition is the composition of the derivatives.

[F6]

Sums, scalar multiples, products and quotients: $(f + g)^{'} (c) = f^{'} (c) + g^{'} (c)$ , $(α f)^{'} (c) = α f^{'} (c)$ , $(f g)^{'} (c) = f^{'} (c) g (c) + f (c) g^{'} (c)$ , and $(f / g)^{'} (c) = (f^{'} (c) g (c) - f (c) g^{'} (c)) / g (c)^{2}$ when $g (c) \neq 0$ : along a coordinate line, the derivative of a product $u v$ is $u^{'} v + u v^{'}$ .

[F7]

Sums, scalar multiples, products, absolute values, maxima, minima and quotients with nonvanishing denominator of continuous functions are continuous, as are constants, the identity and every polynomial function: for real-valued functions on a subset of $R$ , finite sums and products of functions continuous at a point remain continuous there.

[F8]

The Euclidean inner product $⟨ x, y ⟩ = \sum_{k < n} x_{k} y_{k}$ on $R^{n}$ : the Euclidean inner product is the standard dot product and $∥ v ∥_{2}^{2} = ⟨ v, v ⟩$ .

Proof

1.1givenF1F2F4F8algebra

Fix $x \in U$ . By [F1] each component partial of $r_{i}$ exists on a neighborhood and is continuous at $x$ , so [F4] makes $r_{i}$ totally differentiable there with derivative row $\nabla r_{i} (x)^{T}$ . For $h \to 0$ with $x + h \in U$ , let $δ_{i} (h) : = r_{i} (x + h) - r_{i} (x) = \nabla r_{i} (x)^{T} h + o (∥ h ∥_{2})$ ; in particular $δ_{i} (h) = O (∥ h ∥_{2})$ . Expanding each square yields $ϕ (x + h) - ϕ (x) = \sum_{i} r_{i} (x) δ_{i} (h) + \frac{1}{2} \sum_{i} δ_{i} (h)^{2} = (J (x)^{T} r (x))^{T} h + o (∥ h ∥_{2})$ , because the sum is finite and each squared increment is $O (∥ h ∥_{2}^{2})$ . Thus $D ϕ (x) [h] = (J (x)^{T} r (x))^{T} h$ , so the gradient is $J (x)^{T} r (x)$ .

2.1step 1.1F1F2F3F5F6algebra

For coordinates $j, k$ , step 1.1 gives $\partial_{j} ϕ = \sum_{i} r_{i} \partial_{j} r_{i}$ . Restrict each factor to the coordinate line $x + t e_{k}$ and apply the product rule [F6]; the chain rule [F5] identifies the derivatives of the restrictions with the corresponding partial derivatives. Therefore $\partial_{k} \partial_{j} ϕ (x) = \sum_{i} (\partial_{k} r_{i} (x)) (\partial_{j} r_{i} (x)) + \sum_{i} r_{i} (x) \partial_{k} \partial_{j} r_{i} (x)$ . The first matrix has entries $(J^{T} J)_{k j}$ and the second is $\sum_{i} r_{i} \nabla^{2} r_{i}$ , proving the Hessian identity.

3.1F1F7step 1.1step 2.1algebra∎

By [F1], each $r_{i}$ and its first and second partials are continuous on $U$ . To justify continuity of their algebraic combinations on this multivariable domain directly, let $u, v : U \to R$ be continuous at $x$ . Near $x$ , $∣ u (y) ∣ \leq ∣ u (x) ∣ + 1$ , and $∣ u (y) v (y) - u (x) v (x) ∣ \leq (∣ u (x) ∣ + 1) ∣ v (y) - v (x) ∣ + ∣ v (x) ∣ ∣ u (y) - u (x) ∣$ . The right side tends to zero as $y \to x$ ; likewise the difference of a finite sum is bounded by the sum of the absolute differences. These estimates prove the needed sum and product continuity on $U$ , extending the real-domain conclusion of [F7] without applying that theorem outside its domain. They show that $ϕ$ and the first and second partials displayed in steps 1.1 and 2.1 are continuous. Thus $ϕ$ is $C^{2}$ by [F1].

Depends on

Used by

Dependency tree · two levels

49 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Jorge Nocedal and Stephen J. Wright, Numerical Optimization, second edition (standard reference, not scraped)
David Bindel, CS 4220: Numerical Analysis, Gauss-Newton and nonlinear least squares (standard reference, not scraped)