Multiclass margins, ramp loss, and empirical margin loss

Let $K\ge 2$, let $D$ be a Borel probability measure on ${\mathbb{R}}^d\times \{1,\dots ,K\}$, and let $F:{\mathbb{R}}^d\to {\mathbb{R}}^K$ be a continuous score map. At $(x,y)$ its multiclass margin is $m_F(x,y):=F(x{)}_y-{\max }_{j\mathrm{\ne }y}F(x{)}_j\mathrm{.}$ The predicted label is ${\widehat{y}}_F(x):=\min \{j\in \{1,\dots ,K\}:F(x{)}_j={\max }_{1\le l\le K}F(x{)}_l\}\mathrm{.}$ Its population zero-one error is $L_0(F):=D(\{(x,y):{\widehat{y}}_F(x)\mathrm{\ne }y\})\mathrm{.}$ For $\gamma >0$, the ramp loss is ${\mathrm{\ell }}_{\gamma }(r):=\{\begin{array}{ll}0,& r<-\gamma ,\\ 1+r\mathrm{/}\gamma ,& -\gamma \le r\le 0,\\ 1,& r>0.\end{array}$ The population ramp risk is $R_{\gamma }(F):=\int {\mathrm{\ell }}_{\gamma }(-m_F(x,y))D(d(x,y))$. For a sample $S=((x_1,y_1),\dots ,(x_n,y_n))$ of length $n\ge 1$, the empirical ramp risk and empirical margin loss are ${\widehat{R}}_{\gamma ,S}(F):=\frac{1}{n}{\sum }_{i=1}^n{\mathrm{\ell }}_{\gamma }(-m_F(x_i,y_i)),{\widehat{L}}_{\gamma ,S}(F):=\frac{1}{n}{\sum }_{i=1}^n\mathbf{1}\{m_F(x_i,y_i)\le \gamma \}\mathrm{.}$ The score coordinates are continuous, and finite maxima, comparisons, and the piecewise-linear ramp preserve Borel measurability. Thus the classification event and ramp loss in the displayed integrals are Borel sets or functions, so the risks are well-defined for $D$.

The Borel sigma-algebra of a topological space: Borel subsets are members of the sigma-algebra generated by open sets.

Probability measures and probability spaces: a probability measure is a measure of total mass one.

Supervised prediction problems: fixes measurable instance and label spaces, a joint law, predictors, and measurable losses.

Population and empirical risk: defines population risk as an extended nonnegative integral and empirical risk as a finite sample average.

Bartlett, Foster, and Telgarsky, §3.1 and Appendix A.2: gives the multiclass margin operator and the ramp convention.

Mohri, Lecture 4, pp. 28–31: treats confidence margin, empirical margin loss, and margin-based risk bounds for binary scores.