DefinitionDefinition: Literature-sourcedProof: Not applicablePipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Homogeneous classifier margins and exponential training risk

Definition

Let $d, p \geq 1$ , let $X \subseteq R^{d}$ be an input set, and let $Φ : R^{p} \times X \to R$ be the scalar output of a finite feedforward computation graph: the graph is finite and acyclic, its vertices are evaluated in a fixed topological order, and each non-input value is a scalar activation applied to a weighted sum of earlier values and a bias. For a finite sample $(x_{i}, y_{i})_{i = 1}^{n}$ with $n \geq 1$ , $x_{i} \in X$ , and $y_{i} \in {- 1, + 1}$ , define the signed margin $q_{i} (θ) : = y_{i} Φ (θ; x_{i})$ , the minimum margin $q_{\min} (θ) : = \min_{1 \leq i \leq n} q_{i} (θ)$ , and the exponential empirical risk $L (θ) : = \sum_{i = 1}^{n} e^{- q_{i} (θ)}$ . The score map is positively homogeneous of degree $L > 0$ in its parameters if $Φ (c θ; x) = c^{L} Φ (θ; x)$ for every $c > 0$ , $θ \in R^{p}$ , and $x \in X$ ; this then gives $q_{i} (c θ) = c^{L} q_{i} (θ)$ . For the gradient-flow results, assume each sample margin is locally Lipschitz, and let $\partial^{\circ} q_{i} (θ)$ denote the closed convex hull of limits of ordinary gradients at differentiability points approaching $θ$ . Assume also that along every absolutely continuous arc $θ (t)$ , $\frac{d}{d t} q_{i} (θ (t)) = ⟨ h, \dot{θ} (t) ⟩$ for every $h \in \partial^{\circ} q_{i} (θ (t))$ at almost every $t$ . For $θ \neq 0$ , the normalized margin is $\overset{ˉ}{γ} (θ) : = q_{\min} (θ) / ∥ θ ∥_{2}^{L}$ . When $θ \neq 0$ and $L (θ) < 1$ , define the smoothed normalized margin by $\tilde{γ} (θ) : = \log (1 / L (θ)) / ∥ θ ∥_{2}^{L}$ .

Definition references

Finite acyclic feedforward graph and node-evaluation convention

Clarke subdifferential and chain-rule convention

Kaifeng Lyu and Jian Li, Gradient Descent Maximizes the Margin of Homogeneous Neural Networks, §§3–4

Matus Telgarsky, Deep Learning Theory lecture notes, §§10.1–10.2

Depends on

Used by

Dependency tree · one level

2 results within one dependency step of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Kaifeng Lyu and Jian Li, Gradient Descent Maximizes the Margin of Homogeneous Neural Networks, §§3–4 (standard reference, not scraped)
Matus Telgarsky, Deep Learning Theory lecture notes, §§10.1–10.2 (standard reference, not scraped)