Alphabeta MathMirror descent and adaptive regularization
About this page
How statement and proof provenance work
The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.
- Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
- AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
- AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.
These labels describe origin, not correctness: citations and verification chips remain separate evidence.
1 · Prerequisites
Abstract Algebra
- Algebraic Closure, Embeddings, and Separability
- Algebraic Extensions, Extension Degree, and Finite Fields
- Binary Operations, Monoids, Groups and Subgroups
- Composition Series, the Jordan–Hölder Theorem and Solvable Groups
- Cosets, Index and Lagrange's Theorem
- Cyclic Groups and Direct Products
- Divisibility, Euclidean Domains, Principal Ideal Domains and Unique Factorisation
- Group Actions, Orbits, Stabilisers and Cayley's Theorem
- Group Homomorphisms and the Isomorphism Theorems
- Ideals, Quotient Rings and the Isomorphism Theorems for Rings
- Normal Subgroups and Quotient Groups
- Polynomial Rings, the Division Algorithm and Roots
- Rings, Subrings, Integral Domains and Fields
- Simple Field Extensions and the Construction of the Complex Numbers
- Splitting Fields
- Sylow's Theorems, p-Groups and Nilpotent Groups
- The Fundamental Theorem of Algebra
- The Fundamental Theorem of Finite Abelian Groups
- The Galois Correspondence
Linear Algebra
- Determinants of Matrices over a Commutative Ring
- Diagonalisation and the Minimal Polynomial
- Dual Spaces, Bilinear and Quadratic Forms, and Sylvester's Law of Inertia
- Eigenvalues, Eigenvectors and the Characteristic Polynomial
- Inner Product Spaces, Gram-Schmidt, Projections and Adjoints
- Linear Independence, Bases and Dimension
- Linear Transformations, Rank-Nullity and Quotient Spaces
- Matrices, the Matrix of a Linear Map, and Change of Basis
- The Determinant of a Linear Operator, Cofactors and Cramer's Rule
- The Moore--Penrose Pseudoinverse and Regularised Least Squares
- The Spectral Theorem, Positive Operators and Singular Value Decomposition
- Triangularisation, Generalised Eigenspaces and Jordan Canonical Form
- Vector Spaces, Linear Subspaces, Span and Direct Sums
Machine Learning Algorithms
Measure Theory
Number Theory
Numerical Optimization Inverse Problems and Uncertainty Quantification
Real Analysis
- Absolute and Conditional Convergence; Rearrangement; Products
- Construction of the Real Numbers via Cauchy Sequences
- Construction of the Real Numbers via Dedekind Cuts
- Continuity, IVT, EVT, and Uniform Continuity
- Convex and Semicontinuous Functions on Rⁿ
- Convexity
- Countability and Uncountability
- Darboux, L'Hôpital, and Taylor's Theorem
- Foundations of the Real Numbers for Analysis
- Limits of Real Functions
- limsup, liminf, and Subsequential Limits
- Mixed Partials, Taylor Formulae, and Extrema
- Monotone Functions, Discontinuities, and Continuity Sets
- Monotone Sequences, Bolzano-Weierstrass, and Cauchy Completeness
- Power Series and Real-Analytic Functions
- Properties of the Integral and the Working FTC
- Rⁿ as a Normed Space; Vector-Valued Functions
- Roots, Rational Powers, and Classical Inequalities
- Sequences and Limits
- Sequences and Series of Functions; Uniform Convergence
- Series: Convergence and the Nonnegative Tests
- Suprema and Infima
- The Derivative and the Mean Value Theorems
- The Exponential Function
- The Logarithm and General Powers
- The Riemann Integral: Definition and Integrability
- The Total Derivative in ℝᵐ → ℝⁿ
- Topology of ℝ
Set Theory
2 · Logical flowchart
3 · Items
Definition
Mirror maps, Bregman divergence, and local Hessian norms
Lemma
A positive-semidefinite matrix has a unique positive-semidefinite square root
Lemma
The Bregman three-point identity
Lemma
Positive Hessians make Bregman divergence nonnegative
Lemma
Negative entropy on the simplex is strongly convex and has range log n
Algorithm
Gram-matrix AdaGrad with full or singular diagonal metrics
Lemma
AdaGrad accumulated Gram matrices and both scales are positive semidefinite
Algorithm
Agile online mirror descent
Lemma
A singular AdaGrad metric still has a projection minimizer
Lemma
The diagonal AdaGrad scale is the coordinatewise square root
Lemma
Compactness makes the Bregman projection well-defined
Theorem
A corrected regret bound for agile mirror descent
False statement
Halving both terms in the agile mirror-descent regret bound
Sources
- Elad Hazan, Introduction to Online Convex Optimization, second edition
- Peter Bartlett, Stat 260/CS 294-102: Learning in Sequential Decision Problems, mirror-descent lecture notes
- John Duchi, Elad Hazan, and Yoram Singer, Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
- Duchi, Hazan, and Singer, Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, §2 (AdaGrad background; the finite Gram identity is proved locally here)
- Francesco Orabona, Online Mirror Descent II: Regret and Mirror Version
- Elad Hazan, Introduction to Online Convex Optimization, second edition, Theorem 5.6 and full displayed proof
- Peter Bartlett, Stat 260/CS 294-102: Learning in Sequential Decision Problems, mirror-descent lecture notes, slides 28–30