Approximate empirical risk minimization with a supplied selector

Fix a positive sample size $m$, a nonempty predictor class $\mathcal{F}$, and a finite tolerance $\eta \ge 0$. The input includes a supplied jointly measurable evaluation map $A_{\eta }:(\mathsf{X}\times \mathsf{Y}{)}^m\times \mathsf{X}\to \mathsf{A}$ such that every sample slice $A_{\eta }(S,\cdot )$ belongs to $\mathcal{F}$. For each sample $S$, require that the slice is an exact empirical risk minimizer if ${\widehat{R}}_S^{\ast }(\mathcal{F})=+\mathrm{\infty }$, and otherwise has finite empirical risk at most ${\widehat{R}}_S^{\ast }(\mathcal{F})+\eta$. The procedure returns the predictor $A_{\eta }(S,\cdot )$ on input $S$. Its selector is an explicit input; this procedure asserts no existence or construction of such a selector for an arbitrary class. In particular, $\eta =0$ requires attainment whenever the empirical minimum is finite.

Supervised prediction problems: fixes the measurable sample and predictor spaces.

Population and empirical risk: defines the sample objective.

Empirical risk minimizers: defines exact and approximate ERM without forming $+\mathrm{\infty }+\eta$.

Population and empirical risks are measurable: establishes measurability of the sample objective for a fixed predictor.