DefinitionDefinition: Literature-sourcedProof: Not applicablePipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Probability calibration and calibration error

Definition

Let $K \geq 2$ , let $Y$ be a random variable into the finite label set ${1, \dots, K}$ with its full power-set sigma-algebra on a probability space $(Ω, F, P)$ , and let $G = (G_{1}, \dots, G_{K}) : Ω \to Δ_{K}$ be a measurable forecast, where $Δ_{K} = {p \in [0, 1]^{K} : \sum_{j} p_{j} = 1}$ . The forecast is fully reliable if for every label $j$ and every Borel set $B \subseteq Δ_{K}$ , $P (Y = j, G \in B) = E [1_{{G \in B}} G_{j}] .$ It is classwise calibrated if for every $j$ and every Borel set $B \subseteq [0, 1]$ , $P (Y = j, G_{j} \in B) = E [1_{{G_{j} \in B}} G_{j}] .$ For the top-confidence calibration condition, define $C = \max_{j} G_{j}$ and $\hat{Y} = \min {j : G_{j} = C}$ , so ties go to the smallest label. The forecast is top-confidence calibrated if for every Borel set $B \subseteq [0, 1]$ , $P (Y = \hat{Y}, C \in B) = E [1_{{C \in B}} C] .$ A measurable function $r : Δ_{K} \to Δ_{K}$ is a calibration function for $G$ if for every $j$ and every Borel set $B \subseteq Δ_{K}$ , $P (Y = j, G \in B) = E [1_{{G \in B}} r_{j} (G)] .$ When such a function is supplied and $d : Δ_{K} \times Δ_{K} \to [0, + \infty)$ is measurable, the expected calibration error with discrepancy $d$ is $η_{d} = E [d (r (G), G)]$ .

Definition references

Supervised prediction problems: fixes the measurable instance and label spaces, the joint law, and measurable predictors.

Conditional expectation given a sigma algebra: the event-integral identities in this definition are the conditional-expectation characterization when the indicated versions exist.

Probability measures and probability spaces: a probability law is a measure of total mass one.

The Borel sigma-algebra of a topological space: Borel subsets are generated by the open sets of the simplex's Euclidean subspace topology.

Vaicenavicius et al., full paper PDF: reliability, calibration functions, and expected miscalibration are defined in Sections 2, 3.1, and 3.2.

Depends on

Used by

Conditioning on a feature induces a calibrated forecast Corollary
Binned expected and maximum calibration error Definition
KMR's finite group-conditioned risk-score assignment and calibration Definition
Top-label calibration Definition
A calibrated coarsened risk score loses information Example
A calibrated constant prior forecast Example
A fixed confidence bin hides calibration error Example
Three distinct calibration notions in three classes Example
Fixed-bin calibration limits underestimate for jointly convex discrepancies; refining bins can be consistent Theorem

Dependency tree · two levels

17 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Vaicenavicius, Widmann, Andersson, Lindsten, Roll, and Schön, Evaluating Model Calibration in Classification, PMLR 89 (2019), Sections 2–4 (standard reference, not scraped)