TheoremStatement: Literature-sourcedProof: AI-adaptedPipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Gauss-Newton converges quadratically at a full-rank zero-residual root

Statement

Let $m, n \geq 1$ , and let $r : U \to R^{m}$ be $C^{1}$ on an open $U \subseteq R^{n}$ , and let $x_{*} \in U$ satisfy $r (x_{*}) = 0$ . Let $B = {x : ∥ x - x_{*} ∥_{2} \leq R} \subseteq U$ be convex. Suppose $J = D r$ is $L$ -Lipschitz on $B$ and, for some $σ > 0$ , $∥ J (x) v ∥_{2} \geq σ ∥ v ∥_{2} (x \in B, v \in R^{n}) .$ Thus every $J (x)$ has full column rank. For the exact Gauss-Newton iteration $x_{k + 1} = x_{k} + p_{k}$ , where $p_{k}$ is the unique minimizer of $∥ r (x_{k}) + J (x_{k}) p ∥_{2}^{2}$ . When the iteration uses the exact branch of Gauss-Newton linearized least-squares step with QR/SVD rank safeguards, assume each invocation is supplied with an exact reduced QR factorization of $J (x_{k})$ . Set $C = L / (2 σ)$ . Then every $x_{k} \in B$ satisfies $∥ x_{k + 1} - x_{*} ∥_{2} \leq C ∥ x_{k} - x_{*} ∥_{2}^{2} .$ If $C > 0$ and $∥ x_{0} - x_{*} ∥_{2} \leq \min {R, 1 / (2 C)}$ , all iterates stay in the ball, converge to $x_{*}$ , and contract at least by $1 / 2$ per iteration after the quadratic estimate. If $C = 0$ and $x_{0} \in B$ , one exact step reaches $x_{*}$ . This result concerns exact solves; it asserts no finite-precision convergence rate.

Facts & Assumptions

Given: The residual, root, convex ball, Lipschitz Jacobian, uniform lower bound $σ$ , and exact Gauss-Newton steps in the Statement.

[F1]

Rectangular residual linearization error under a Lipschitz Jacobian: on a convex segment, $∥ r (x) - r (x_{*}) - J (x) (x - x_{*}) ∥_{2} \leq (L / 2) ∥ x - x_{*} ∥_{2}^{2}$ .

[F2]

Gauss-Newton linearized least-squares step with QR/SVD rank safeguards: an exact full-rank branch returns the unique minimizer of the linearized residual least-squares problem.

[F3]

The Euclidean inner product $⟨ x, y ⟩ = \sum_{k < n} x_{k} y_{k}$ on $R^{n}$ : the Euclidean inner product is bilinear and symmetric, and $∥ v ∥_{2}^{2} = v^{T} v$ .

[A1]

$r (x_{*}) = 0$ .

[A2]

$∥ J (x) v ∥_{2} \geq σ ∥ v ∥_{2}$ with $σ > 0$ for every $x \in B$ and every $v$ .

Proof

1.1givenA1F1construct

Fix $x \in B$ , put $e = x - x_{*}$ , and write $E = r (x) - r (x_{*}) - J (x) e$ . By [F1] and [A1], $r (x) = J (x) e + E$ with $∥ E ∥_{2} \leq (L / 2) ∥ e ∥_{2}^{2}$ .

2.1givenA2F2F3step 1.1algebra

Let $p$ be the exact Gauss-Newton step and set $h : = p + e$ . For every $v \in R^{n}$ , minimality of $p$ gives $\frac{1}{2} ∥ r (x) + J (x) (p + t v) ∥_{2}^{2} \geq \frac{1}{2} ∥ r (x) + J (x) p ∥_{2}^{2}$ for all real $t$ . Expanding in $t$ shows the linear coefficient $v^{T} J (x)^{T} (r (x) + J (x) p)$ must be zero (otherwise a sufficiently small $t$ of the opposite sign decreases the quadratic). Since this holds for every $v$ , $J (x)^{T} (J (x) h + E) = 0$ by step 1.1.

3.1step 1.1step 2.1A2F3algebra

Set $q : = - h$ . Step 2.1 gives $J (x)^{T} (E - J (x) q) = 0$ , so $E - J (x) q$ is orthogonal to $J (x) q$ ; hence $∥ E ∥_{2}^{2} = ∥ J (x) q ∥_{2}^{2} + ∥ E - J (x) q ∥_{2}^{2} \geq ∥ J (x) q ∥_{2}^{2}$ . By [A2], $∥ J (x) q ∥_{2} \geq σ ∥ q ∥_{2}$ . Therefore $∥ x + p - x_{*} ∥_{2} = ∥ h ∥_{2} = ∥ q ∥_{2} \leq ∥ E ∥_{2} / σ \leq C ∥ e ∥_{2}^{2}$ .

4.1step 3.1algebra∎

If $C = 0$ and $x_{0} \in B$ , step 3.1 gives $x_{1} = x_{*}$ . If $C > 0$ and $∥ x_{0} - x_{*} ∥_{2} \leq 1 / (2 C)$ , the stated hypothesis also puts $x_{0} \in B$ , and step 3.1 gives $∥ x_{1} - x_{*} ∥_{2} \leq \frac{1}{2} ∥ x_{0} - x_{*} ∥_{2}$ , so $x_{1} \in B$ ; induction repeats the same bound for every $k$ , and $∥ x_{k} - x_{*} ∥_{2} \leq 2^{- k} ∥ x_{0} - x_{*} ∥_{2} \to 0$ .

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

36 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Jorge Nocedal and Stephen J. Wright, Numerical Optimization, second edition (standard reference, not scraped)
David Bindel, CS 4220: Numerical Analysis, Gauss-Newton and nonlinear least squares (standard reference, not scraped)