Alphabeta Math

Representation Learning and Generative Models

34 pages in 5 parts

Representation learning seeks useful features from supervision, reconstruction or relationships among observations. Generative modeling instead specifies or approximates a data distribution and a procedure for sampling or evaluating it. This collection connects these goals while keeping their mathematical objectives distinct. Its starting points are contrastive learning, reconstruction and masked autoencoders, followed by variational latent-variable models. Selected neural architectures, differentiation and statistical-learning results serve as prerequisites from the earlier collections.

Energy models, score matching and adversarial objectives lead into probability metrics and transport duality. Finite-metric duality precedes the compact-metric Kantorovich-Rubinstein result and its Wasserstein-GAN application. Exact-density flows and continuous normalizing flows form a separate route through change of variables and differential equations; flow matching connects transport paths with continuous generation.

Diffusion pages distinguish finite chains, masked discrete generation and continuous score-based sampling. The reverse-time diffusion prerequisite belongs immediately before the score-SDE application. Forward SDE well-posedness, Gaussian smoothing and numerical Euler convergence remain external probability and numerical prerequisites. A solver-error theorem does not control learned-score bias. Multimodal alignment and foundation-model transfer finish the pathway, separating mathematical guarantees from empirical scaling evidence. Online and Reinforcement Learning is a parallel branch; its preference and policy objectives can be consulted for later alignment applications.

Pathway

The parts run in order. Everything a page needs from this group has been read by the time you reach it, and the level on each row is how many dependency steps into the group that page sits.

  1. Part 1 · Representation and latent-variable objectives

    3 pages

    Start with contrastive representations and reconstruction objectives, then study variational autoencoders and pathwise training. Distinguish the objective being optimized from claims about representation quality or identifiability. Relevant neural approximation and variational-inference prerequisites remain in their existing collections.

  2. Part 2 · Energy models, adversarial learning and transport

    5 pages

    Energy and adversarial objectives lead to their distinct divergence and score formulations. Finite-metric transport duality precedes compact-metric Kantorovich-Rubinstein duality, followed by integral probability metrics and Wasserstein GANs. Optimization assumptions and ideal-discriminator identities do not establish successful adversarial training.

  3. Part 3 · Density flows and transport paths

    3 pages

    Begin with exact-density normalizing flows and their change-of-variables assumptions. Continue to neural ODEs and continuous normalizing flows, then conditional flow matching. Continuous trajectories, density integration and numerical error each retain their own regularity and solver prerequisites.

  4. Part 4 · Diffusion and reverse generation

    4 pages

    Study discrete diffusion and masked finite-state generation, then the reverse-time diffusion interface and score-SDE generation. The continuous branch requires the published forward-SDE, Gaussian-smoothing and Euler convergence suppliers. Reverse dynamics, approximation error and learned-model bias remain separate claims.

  5. Part 5 · Multimodal transfer and foundation models

    2 pages

    Multimodal alignment follows its contrastive and neural context; foundation-model transfer follows attention and distribution-shift prerequisites. Keep mathematical statements about objectives and transfer conditions separate from empirical scaling or benchmark observations. Preference and policy optimization form a later bridge to Online and Reinforcement Learning.