ExampleConstruction: Literature-sourcedVerification: AI-adaptedPipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

A bounded periodic gradient sequence with divergent Adam iterates

Statement

For Adam with $β_{1} = 0$ , $β_{2} = 1 / 2$ , $η = 1$ , and $ϵ = 1 / 100$ , feed the repeated affine losses $5 x / 2, - x, - x$ from $x_{1} = 0$ . Every gradient has magnitude at most $5 / 2$ , but the terminal subsequence $x_{3 k + 1}$ tends to $+ \infty$ ; for every $k \geq 2$ , its cycle drift is at least $7295 / 90804$ .

Facts & Assumptions

Given: The update and loss sequence in the Statement.

[F1]

Bias-corrected Adam update: Adam's bias-corrected moments and coordinate update are defined for the fixed parameters in the Statement.

[F2]

Adam diverges on a periodic bounded-gradient loss sequence: this exact loss sequence and parameter choice give cycle drift at least $7295 / 90804 > 0$ for every cycle $k \geq 2$ and hence an unbounded terminal subsequence.

[F3]

Convex and strictly convex functions on Euclidean convex sets: affine losses satisfy the convex segment inequality.

[F4]

Every complete ordered field is Archimedean: for every real $R$ there is a natural integer $N \geq 1$ with $R < N$ .

[F5]

Sign rules for products and monotonicity of multiplication: multiplication by a positive real preserves strict order.

Proof

Given: The finite-prefix online loss sequence in the Statement.

1.1F1F3givenalgebra

Each loss is affine, so direct substitution in [F3] verifies convexity. Its derivative is respectively $5 / 2, - 1, - 1$ , all of magnitude at most $5 / 2$ ; the updates are the bias-corrected Adam updates [F1].

2.1F2F4F5step 1.1givenalgebrainduction∎

The gradients and parameters in step 1.1 are the specified inputs of [F2]. Let $δ = 7295 / 90804 > 0$ ; its cycle calculation gives $x_{3 (k + 1) + 1} - x_{3 k + 1} \geq δ$ for every $k \geq 2$ . With $X_{k} = x_{3 k + 1}$ , induction gives $X_{2 + r} \geq X_{2} + r δ$ for every natural $r \geq 0$ . For any real $M$ , [F4] supplies a natural $N \geq 1$ with $(M - X_{2}) / δ < N$ . Since $δ > 0$ , [F5] gives $X_{2} + N δ > M$ , and then every $r \geq N$ satisfies $X_{2 + r} \geq X_{2} + N δ > M$ . This is exactly convergence of the terminal subsequence to $+ \infty$ , after its first two cycles.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

22 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar, On the Convergence of Adam and Beyond, ICLR 2018, full arXiv text (standard reference, not scraped)
Diederik P. Kingma and Jimmy Ba, Adam: A Method for Stochastic Optimization, ICLR 2015 (standard reference, not scraped)