Alphabeta Math

Statistical Learning and Classical Models

68 pages in 5 parts

Statistical learning studies how predictions are defined, fitted and evaluated from data. This collection begins with loss, risk and empirical minimization, then develops finite-class guarantees, VC dimension, Rademacher complexity, stability, compression and PAC-Bayes bounds. These results distinguish fitting the observed sample from controlling performance on new observations. Probability, mathematical statistics and linear algebra provide the external foundations; the learning-specific guarantees retain their own sampling and measurability assumptions.

The modeling pathway covers linear and sparse prediction, classification, kernels, Gaussian processes, trees and ensembles. It continues through clustering, spectral methods, dimension reduction, latent factors, graphical models and causal discovery. Active querying and semi-supervised methods explain what additional information a learning protocol can use, and what unlabeled data alone cannot identify. Approximate kernel solves retain their numerical error contracts.

The final parts address distribution shift, adversarial risk, calibration, fairness, deployment guarantees, privacy and federated learning. These are distinct mathematical questions: calibration does not imply accuracy or fairness, and privacy does not establish generalization by itself. Read the relevant theory and model prerequisites before each application rather than treating the whole collection as a compulsory linear course. Neural Networks and Learning Dynamics builds on selected parts here; online decision methods can branch from the same classical foundations.

Pathway

The parts run in order. Everything a page needs from this group has been read by the time you reach it, and the level on each row is how many dependency steps into the group that page sits.

  1. Part 1 · Risk, complexity and generalization

    8 pages

    Start with prediction problems and risk, then develop PAC guarantees, VC dimension, Rademacher complexity, stability, compression and minimax limits. These pages provide the language for the later model families. Their guarantees retain the stated sampling, boundedness and measurability conditions.

  2. Part 2 · Prediction, kernels and ensembles

    9 pages

    Develop linear and sparse prediction, classifiers, kernel regularization and Gaussian-process prediction. Trees lead to bagging and boosting; nearest-neighbor methods provide a complementary geometric viewpoint. Read the kernel and Gaussian-process interfaces before the certified approximate-solve extension.

  3. Part 3 · Clustering, latent structure and causal discovery

    8 pages · after Part 1

    Continue through mixtures, spectral clustering, dimension reduction, latent factors and graphical structure. The causal-discovery pages distinguish observational equivalence from identifiability. Semi-supervised graph propagation follows its spectral and risk prerequisites and makes its nonidentification limits explicit.

  4. Part 4 · Shift, calibration and deployment

    6 pages · after Part 1

    Use the earlier risk and generalization results to study distribution shift, robustness and fairness. Predictive calibration precedes monotone risk calibration and deployment guarantees. Active querying draws on the earlier PAC and VC results; its feedback protocol differs from passive learning.

  5. Part 5 · Privacy, private learning and federation

    3 pages · after Parts 1 and 2

    Begin with privacy mechanisms and composition, then study private empirical minimization and federated optimization. These methods use the earlier classical models and statistical guarantees. They supply context for the stochastic-training stability results in Neural Networks and Learning Dynamics.