Validation temperature scaling

Fix $K\ge 2$. The input is a trained classifier with a measurable, finite-valued logit map $z:\mathcal{X}\to {\mathbb{R}}^K$, a nonempty finite validation sample $(x_i,y_i{)}_{i=1}^{n_v}$ with $n_v\ge 1$ and $y_i\in \{1,\dots ,K\}$, and a finite nonempty set of candidate temperatures $\mathcal{T}\subset (0,\mathrm{\infty })$. The validation labels were not used to fit the logits. For $T\in \mathcal{T}$, compute $p_T(j\mid x)=\frac{\exp (z_j(x)\mathrm{/}T)}{{\sum }_{\mathrm{\ell }=1}^K\exp (z_{\mathrm{\ell }}(x)\mathrm{/}T)}(j=1,\dots ,K),L_{\mathrm{v}\mathrm{a}\mathrm{l}}(T)=-\frac{1}{n_v}{\sum }_{i=1}^{n_v}\log p_T(y_i\mid x_i)\mathrm{.}$ Choose the smallest temperature among the minimizers, $T^{\ast }=\min \arg {\min }_{T\in \mathcal{T}}{L}_{\mathrm{v}\mathrm{a}\mathrm{l}}(T)\mathrm{.}$ and return the forecasts $p_{T^{\ast }}(\cdot \mid x)$ and predicted label $\min \arg {\max }_j{p}_{T^{\ast }}(j\mid x)$ for each input $x$. This is a finite-candidate version of validation negative-log-likelihood temperature scaling. It selects only by empirical validation loss; it makes no finite-sample calibration guarantee for future data, particularly when the validation and future data laws differ.

Strictly proper scoring rules on a finite label set: defines the finite-label logarithmic loss used by the validation objective.

Log and Brier losses are strictly proper: establishes that log loss is strictly proper; this does not itself give a finite-sample calibration guarantee.

The natural logarithm as the inverse of the exponential function: fixes the logarithm and its domain on positive reals.

The real exponential function and the number $e$ by a power series: defines the exponential used in the softmax formula.

A measurable function between measurable spaces: specifies the measurability condition on the logit map.

Positive temperature scaling preserves the maximizing class ↗: proves normalization, finite validation loss, existence of the finite-grid minimizer, and preservation of the maximizing class.

Guo et al., full paper, Section 4.2: introduces multiclass temperature scaling, validation NLL selection, and preservation of the class prediction.