ExampleConstruction: AI-generatedVerification: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

A fixed confidence bin hides calibration error

Example

Give $X \in {a, b}$ and $Y \in {1, 2}$ their full power-set sigma-algebras. Let $X$ be uniform on ${a, b}$ , let $P (Y = 1 ∣ X = a) = 0.8$ and $P (Y = 1 ∣ X = b) = 0.6$ , and forecast $G (a) = (0.6, 0.4)$ , $G (b) = (0.8, 0.2)$ . For an iid sample from this law and discrepancy $d (p, q) = ∣ p_{1} - q_{1} ∣$ , the full calibration error is $0.2$ . If the entire forecast simplex is treated as one bin, the empirical binned calibration estimate converges almost surely to zero.

Example references

Probability calibration and calibration error: defines the calibration function and expected calibration error.

Fixed-bin calibration limits underestimate for jointly convex discrepancies; refining bins can be consistent: fixed-bin estimates converge almost surely to their population bin value, which can be strictly below calibration error.

Facts & Assumptions

Given: The two-point feature-label law, forecast, discrepancy, and one-bin partition stated in the example.

[F1]

The calibration function on a finite forecast support equals the conditional class-probability vector at each forecast value. Probability calibration and calibration error

[F2]

For a fixed finite data-independent partition, the empirical estimator converges almost surely to the corresponding population binned value. Fixed-bin calibration limits underestimate for jointly convex discrepancies; refining bins can be consistent

[F3]

The sample averages of the bounded label and forecast coordinates converge almost surely to their means. Kolmogorov iid l1 strong law

[F4]

$d (p, q) = ∣ p_{1} - q_{1} ∣$ is a continuous, jointly convex, nonnegative discrepancy with $d (p, p) = 0$ .

Verification

1.1givenF1algebra

The forecast values identify the two feature values, so the calibration function is $r (0.6, 0.4) = (0.8, 0.2)$ and $r (0.8, 0.2) = (0.6, 0.4)$ . Therefore $E [d (r (G), G)] = \frac{1}{2} ∣ 0.8 - 0.6 ∣ + \frac{1}{2} ∣ 0.6 - 0.8 ∣ = 0.2$ .

2.1F2F3F4step 1.1givenalgebra∎

In the single bin $B = Δ_{2}$ , the mean forecast first coordinate is $(0.6 + 0.8) / 2 = 0.7$ , and the mean label-indicator first coordinate is $(0.8 + 0.6) / 2 = 0.7$ . Hence the population binned discrepancy is $∣ 0.7 - 0.7 ∣ = 0$ . By [F2] the empirical one-bin estimate converges almost surely to zero; equivalently, [F3] gives convergence of both sample averages to $0.7$ .

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

17 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Vaicenavicius et al., Evaluating Model Calibration in Classification, PMLR 89 (2019), Theorems 1–2 (standard reference, not scraped)
Rick Durrett, Probability: Theory and Examples, 5th ed., Section 2.4 (standard reference, not scraped)