TheoremStatement: Literature-sourcedProof: AI-adaptedPipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Gauss-Newton contracts linearly under a controlled residual-curvature bound

Statement

Let $m, n \geq 1$ , and let $r : U \to R^{m}$ be $C^{2}$ on an open $U \subseteq R^{n}$ , let $ϕ = ∥ r ∥_{2}^{2} / 2$ , $J = D r$ , $A = J^{T} J$ , and $S (x) = \sum_{i} r_{i} (x) \nabla^{2} r_{i} (x)$ . Let $x_{*} \in U$ satisfy $J (x_{*})^{T} r (x_{*}) = 0$ , and let the closed ball $B = {x : ∥ x - x_{*} ∥_{2} \leq R}$ lie in $U$ , with $R > 0$ . Suppose on $B$ :

$J (x)$ has full column rank and smallest singular value at least $σ > 0$ ;
$∥ J (x) ∥_{2} \leq M$ ;
$J$ is $L$ -Lipschitz in the Euclidean induced matrix norm;
$∥ S (x) ∥_{2} \leq κ$ .

Define $q = (κ + M L R) / σ^{2}$ and assume $q < 1$ . For the exact full-rank Gauss-Newton step $p (x) = - A (x)^{- 1} J (x)^{T} r (x)$ , supply an exact reduced QR factorization of $J (x)$ to the exact branch of Gauss-Newton linearized least-squares step with QR/SVD rank safeguards at each $x \in B$ ; the full-column-rank hypothesis guarantees such a factorization. Every $x \in B$ satisfies $∥ x + p (x) - x_{*} ∥_{2} \leq q ∥ x - x_{*} ∥_{2} .$ Consequently the exact iterates from any $x_{0} \in B$ stay in $B$ and converge to $x_{*}$ at a linear rate no larger than $q$ . The curvature hypothesis controls the residual term $S$ ; no quadratic rate is claimed when it is nonzero.

Facts & Assumptions

Given: The residual map, stationary point, ball, and quantitative bounds in the Statement.

[F1]

The nonlinear least-squares gradient and Hessian formulas: $\nabla ϕ = J^{T} r$ and $\nabla^{2} ϕ = J^{T} J + S$ .

[F2]

If $f : [a, b] \to R^{m}$ is differentiable with integrable $f^{'}$ then $\int_{a}^{b} f^{'} = f (b) - f (a)$ ; and a bounded derivative makes $f$ Lipschitz: for a vector-valued path differentiable on a closed interval with integrable derivative, the endpoint difference is the integral of that derivative.

[F3]

Induced matrix norms are compatible with matrix-vector multiplication, submultiplicative, and satisfy ||I|| = 1: the induced Euclidean matrix norm satisfies $∥ A v ∥_{2} \leq ∥ A ∥_{2} ∥ v ∥_{2}$ and $∥ A B ∥_{2} \leq ∥ A ∥_{2} ∥ B ∥_{2}$ .

[F4]

Every linear map between finite-dimensional real or complex inner product spaces admits a singular value decomposition and Spectral and Frobenius norms are unitarily invariant, are given by singular values, and satisfy the sharp rank comparison: $J = U_{1} Σ V^{T}$ on its full-column-rank subspace, and orthogonal factors preserve the operator norm.

[F5]

The matrix norm induced by a published vector p-norm: the matrix $2$ -norm is induced by the Euclidean vector norm.

[F6]

The chain rule for total derivatives: $D (g \circ f) (a) = D g (f (a)) \circ D f (a)$ : the derivative of the path composition is the derivative of the outer map applied to its tangent.

[F7]

For $a \leq b$ and $f : [a, b] \to R^{m}$ integrable when $a < b$ , $∥ \int_{a}^{b} f ∥_{2} \leq \int_{a}^{b} ∥ f ∥_{2}$ ; for $a < b$ , $∥ f ∥_{2}$ is integrable: the norm of an integrable vector-valued integral is at most the integral of the pointwise norm.

[F8]

The Euclidean inner product $⟨ x, y ⟩ = \sum_{k < n} x_{k} y_{k}$ on $R^{n}$ : the Euclidean norm is induced by the standard dot product and $∥ v ∥_{2}^{2} = v^{T} v$ .

[A1]

$x_{*}$ is stationary: $J (x_{*})^{T} r (x_{*}) = 0$ .

[A2]

The bounds and strict contraction factor $q < 1$ hold throughout the stated convex ball.

Proof

1.1givenA2F4F5F8algebra

For each $x \in B$ , the SVD [F4] writes $J (x) = U_{1} Σ V^{T}$ with every diagonal entry of $Σ$ at least $σ$ . Thus $A (x) = V Σ^{2} V^{T}$ is invertible and $A (x)^{- 1} = V Σ^{- 2} V^{T}$ . By [F4]–[F5], $∥ A (x)^{- 1} ∥_{2} \leq σ^{- 2}$ .

1.2givenA1F1F2F6construct

Fix $x \in B$ , put $e = x - x_{*}$ , and set $x_{t} = x_{*} + t e$ . The path $t \mapsto \nabla ϕ (x_{t})$ is continuously differentiable; [F1] and [F6] give derivative $(A (x_{t}) + S (x_{t})) e$ . This derivative is continuous and hence componentwise integrable, so [F2] and [A1] yield $J (x)^{T} r (x) = \int_{0}^{1} (A (x_{t}) + S (x_{t})) e d t$ .

1.3givenA2F3F4algebra

For $0 \leq t \leq 1$ , write $A (x) - A (x_{t}) = J (x)^{T} (J (x) - J (x_{t})) + (J (x)^{T} - J (x_{t})^{T}) J (x_{t})$ . An SVD of $J (x) - J (x_{t})$ also gives an SVD of its transpose with the same singular values, so [F4] gives $∥ J (x)^{T} - J (x_{t})^{T} ∥_{2} = ∥ J (x) - J (x_{t}) ∥_{2}$ and $∥ J (x)^{T} ∥_{2} = ∥ J (x) ∥_{2}$ . The Lipschitz and boundedness hypotheses, with [F3], now give $∥ A (x) - A (x_{t}) ∥_{2} \leq 2 M L (1 - t) ∥ e ∥_{2}$ .

2.1step 1.1step 1.2step 1.3A2F3F7F8algebra

The exact step satisfies $x + p (x) - x_{*} = A (x)^{- 1} (A (x) e - J (x)^{T} r (x))$ . Step 1.2 makes the parenthesized term $\int_{0}^{1} (A (x) - A (x_{t}) - S (x_{t})) e d t$ . By step 1.3, $∥ S (x_{t}) ∥_{2} \leq κ$ , and [F7] bounds the integral norm by $(M L ∥ e ∥_{2} + κ) ∥ e ∥_{2}$ . Step 1.1 therefore gives $∥ x + p (x) - x_{*} ∥_{2} \leq σ^{- 2} (M L ∥ e ∥_{2} + κ) ∥ e ∥_{2} \leq q ∥ e ∥_{2}$ , since $∥ e ∥_{2} \leq R$ .

3.1step 2.1algebra∎

Because $q < 1$ , step 2.1 keeps every iterate in $B$ and gives $∥ x_{k} - x_{*} ∥_{2} \leq q^{k} ∥ x_{0} - x_{*} ∥_{2}$ , which tends to zero.

Depends on

Used by

A nonzero-residual stationary point gives only linear Gauss-Newton convergence Example

Dependency tree · two levels

84 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Jorge Nocedal and Stephen J. Wright, Numerical Optimization, second edition (standard reference, not scraped)
David Bindel, CS 4220: Numerical Analysis, Gauss-Newton and nonlinear least squares (standard reference, not scraped)