Pipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Momentum and quadratic acceleration — Examples

1 · Prerequisites

2 · Logical flowchart

3 · Definitions, theorems and proofs

None yet.

4 · Examples, counterexamples and false statements

ExampleConstruction: AI-generatedVerification: AI-generatedOpen item page →

An untuned heavy-ball recurrence diverges on a scalar quadratic

Example

Let $F : R \to R$ be $F (x) = \frac{1}{2} x^{2}$ . Run heavy-ball momentum with $α = \frac{27}{8}$ , $β = \frac{1}{4}$ , and $w_{0} = w_{1} = 1$ . For every integer $t \geq 0$ , the resulting iterates are

$w_{t} = - \frac{3}{5} (- 2)^{t} + \frac{8}{5} {(- \frac{1}{8})}^{t},$

so the iterate sequence is unbounded and does not converge to a real limit.

Example references

Heavy-ball momentum

Matrix quadratic forms have the expected first derivative and Hessian

Completeness of the real numbers

Every complete ordered field is Archimedean

Facts & Assumptions

Given: $F (x) = \frac{1}{2} x^{2}$ , heavy-ball parameters $α = \frac{27}{8}$ and $β = \frac{1}{4}$ , and initial values $w_{0} = w_{1} = 1$ .

[F1]

Heavy-ball momentum: for fixed parameters, $w_{t + 1} = w_{t} - α \nabla F (w_{t}) + β (w_{t} - w_{t - 1})$ .

[F2]

Matrix quadratic forms have the expected first derivative and Hessian: for $q (x) = \frac{1}{2} ∥ C x ∥_{2}^{2}$ , the gradient is $C^{*} C x$ .

[F3]

The reals are complete: with its usual order and field structure, $R$ is a complete ordered field.

[F4]

Every complete ordered field is Archimedean: in every complete ordered field, for every element $M$ there is a canonical natural number $n \geq 1$ whose field embedding satisfies $M < n$ .

Verification

1.1givenF1F2algebra

Taking $C = [1]$ in [F2] gives $\nabla F (x) = x$ . Substituting the parameters and this gradient into [F1] yields $w_{t + 1} = - \frac{17}{8} w_{t} - \frac{1}{4} w_{t - 1}$ for every $t \geq 1$ .

2.1givenstep 1.1algebrainduction

The characteristic polynomial is $r^{2} + \frac{17}{8} r + \frac{1}{4} = (r + 2) (r + \frac{1}{8})$ . Each geometric sequence $(- 2)^{t}$ and $(- \frac{1}{8})^{t}$ therefore satisfies the recurrence in step 1.1. The stated linear combination has $w_{0} = 1$ and $w_{1} = \frac{6}{5} - \frac{1}{5} = 1$ , so the recurrence and these initial values prove the displayed formula for all $t$ by induction.

3.1givenF3F4step 2.1algebrainduction∎

For $t \geq 1$ , factor the expression as $w_{t} = (- 2)^{t} (- \frac{3}{5} + \frac{8}{5} 16^{- t})$ . Since $16^{- t} \leq 1 / 16$ , the term in parentheses is at most $- \frac{1}{2}$ , hence $∣ w_{t} ∣ \geq 2^{t - 1}$ . Induction gives $2^{t - 1} \geq t$ for $t \geq 1$ . For any starting index $N \geq 1$ and any real $R > 0$ , put $B = \max {N, R}$ ; since $R$ is a complete ordered field by [F3], [F4] supplies a natural number $t > B$ , which we use under the usual embedding as an integer iteration index. Thus $t \geq N$ and $∣ w_{t} ∣ \geq t > R$ . Every tail is therefore unbounded. If the sequence converged to a real $ℓ$ , some tail would satisfy $∣ w_{t} - ℓ ∣ < 1$ and hence $∣ w_{t} ∣ < ∣ ℓ ∣ + 1$ , a contradiction.

ExampleConstruction: Literature-sourcedVerification: Not suppliedjudge pass (gpt-6-luna)Open item page →

A reported stochastic Nesterov experiment is empirical evidence

Example

Sutskever, Martens, Dahl, and Hinton report experiments on the Curves, MNIST, and Faces deep-autoencoder tasks using sigmoid networks, their sparse initialization, 750,000 parameter updates per trial, minibatches of size 200, and no regularization. They used a slowly increasing momentum schedule capped by $μ_{\max}$ and selected the best training learning rate for each schedule from ${0.05, 0.01, 0.005, 0.001, 0.0005, 0.0001}$ . In Table 1, the Curves training squared error is $0.074$ for their Nesterov variant with $μ_{\max} = 0.999$ and $0.48$ for the zero-momentum SGD baseline. The paper reports training errors only; this setup-specific empirical comparison supplies no general stochastic convergence rate or test-risk guarantee.

Example references

Nesterov accelerated gradient algorithm The deterministic algorithm evaluates the gradient at the look-ahead point; the reported experiment replaces the full gradient by a minibatch gradient.

Sutskever, Martens, Dahl, and Hinton, On the Importance of Initialization and Momentum in Deep Learning

Bottou, Curtis, and Nocedal, Optimization Methods for Large-Scale Machine Learning

ExampleConstruction: AI-generatedVerification: AI-generatedjudge pass (gpt-6-luna)Open item page →

The spectrum 1, 4, 9 gives tuned root factor one half

Example

Let $A = diag (1, 4, 9)$ , $F (w) = \frac{1}{2} w^{T} A w$ on $R^{3}$ , and $w_{0} = w_{1} = (1, 1, 1)$ . With $c = 1$ and $L = 9$ , tuned heavy-ball has $α = \frac{1}{4}$ , $β = \frac{1}{4}$ , and modal root factor $\frac{1}{2}$ at all three eigenvalues. Its endpoint characteristic polynomials are repeated-root polynomials. Gradient descent with step $γ = \frac{1}{5}$ has modal multipliers $\frac{4}{5}, \frac{1}{5}, - \frac{4}{5}$ , so its worst modal factor is $\frac{4}{5}$ .

Example references

Tuned heavy-ball has the minimax root factor on positive-definite quadratics

Heavy-ball momentum

Gradient descent

Facts & Assumptions

Given: The diagonal matrix $A = diag (1, 4, 9)$ , objective $F (w) = \frac{1}{2} w^{T} A w$ , initial vector $w_{0} = w_{1} = (1, 1, 1)$ , and the two fixed-step update rules.

[F1]

Tuned heavy-ball has the minimax root factor on positive-definite quadratics: for $c = 1$ , $L = 9$ , the tuned parameters are $α = 4 / (\sqrt{L} + \sqrt{c})^{2}$ and $β = ((\sqrt{L} - \sqrt{c}) / (\sqrt{L} + \sqrt{c}))^{2}$ , with modal root factor $q = (\sqrt{L} - \sqrt{c}) / (\sqrt{L} + \sqrt{c})$ .

[F2]

Heavy-ball momentum: the update is $w_{t + 1} = w_{t} - α \nabla F (w_{t}) + β (w_{t} - w_{t - 1})$ .

[F3]

Gradient descent: in fixed-step mode, the update is $x \mapsto x - γ \nabla F (x)$ .

[F4]

Matrix quadratic forms have the expected first derivative and Hessian: for $q (x) = \frac{1}{2} ∥ C x ∥_{2}^{2}$ , the gradient is $C^{*} C x$ .

Verification

1.1givenF1algebra

The eigenvalues of $A$ are $1, 4, 9$ , so $c = 1$ , $L = 9$ , $κ = 9$ , and $q = (3 - 1) / (3 + 1) = 1 / 2$ . Substitution into [F1] gives $α = 4 / (3 + 1)^{2} = 1 / 4$ and $β = q^{2} = 1 / 4$ .

1.2givenF2F4algebra

Taking $C = diag (1, 2, 3)$ gives $C^{T} C = A$ , so [F4] yields $\nabla F (w) = A w$ . Projecting [F2]'s update onto each standard coordinate gives polynomial $p_{λ} (r) = r^{2} - (1 + β - α λ) r + β$ . For $λ = 1$ it is $r^{2} - r + 1 / 4 = (r - 1 / 2)^{2}$ ; for $λ = 4$ it is $r^{2} - r / 4 + 1 / 4$ , whose roots are $1 / 8 \pm i \sqrt{15} / 8$ and have modulus $1 / 2$ ; for $λ = 9$ it is $r^{2} + r + 1 / 4 = (r + 1 / 2)^{2}$ . Thus each root factor is $1 / 2$ , and the endpoint roots are repeated.

2.1givenF3algebra∎

By [F3], gradient descent with $γ = 1 / 5$ has multiplier $1 - λ / 5$ on an eigenvector of eigenvalue $λ$ . The multipliers at $1, 4, 9$ are $4 / 5, 1 / 5, - 4 / 5$ , whose largest absolute value is $4 / 5$ .

Momentum and quadratic acceleration — Examples

2 · Logical flowchart

3 · Definitions, theorems and proofs

4 · Examples, counterexamples and false statements

An untuned heavy-ball recurrence diverges on a scalar quadratic

Example

Example references

Facts & Assumptions

Verification

A reported stochastic Nesterov experiment is empirical evidence

Example

Example references

The spectrum 1, 4, 9 gives tuned root factor one half

Example

Example references

Facts & Assumptions

Verification

Sources