CounterexampleConstruction: AI-generatedVerification: AI-generatedPipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

A nonmonotone loss can defeat first-pass calibration

Statement refuted

For every deterministic grid $Λ = {λ_{1} < \dots < λ_{M}}$ with $M \geq 1$ , calibration size $n \geq 1$ , bound $B > 0$ , and target $α \in [0, B)$ , suppose a probability space carries $n + 1$ exchangeable random elements $W_{i} = (L_{i} (λ_{j}))_{j = 1}^{M}$ of the Borel space $[0, B]^{M}$ , with $L_{i} (λ_{M}) \leq α$ almost surely for every $i$ . The B-corrected first-passing rule then has expected test loss $E [L_{n + 1} (\hat{λ})] \leq α$ , even when the losses are not nonincreasing along the grid.

Statement references

Finite-grid conformal risk control: the first-passing rule and maximum-grid fallback.

Exchangeable random elements: joint-law invariance under all row permutations.

Angelopoulos et al., Conformal Risk Control, §2.4

Facts & Assumptions

Given: The refuted assertion and the following finite model: $n = 1$ , $M = 2$ , $B = 1$ , $α = 3 / 4$ , and $λ_{1} < λ_{2}$ . Let $Z_{1}, Z_{2}$ be independent with $P (Z_{i} = a) = 1 / 10$ and $P (Z_{i} = b) = 9 / 10$ . Define the losses by $L_{i} (λ_{1}) = 0$ and $L_{i} (λ_{2}) = 3 / 4$ when $Z_{i} = a$ , and by $L_{i} (λ_{1}) = 1$ and $L_{i} (λ_{2}) = 3 / 4$ when $Z_{i} = b$ .

[F1]

Exchangeability means invariance of the joint law under every coordinate permutation. Exchangeable random elements

[F2]

The algorithm tests $(L_{1} (λ_{j}) + B) / (n + 1)$ at each grid point in increasing order and returns the maximum point if no test passes. Finite-grid conformal risk control

[A1]

The two rows are independent draws from the same two-point law specified above, so their joint law is exchangeable.

[A2]

The loss table has values in $[0, 1]$ and endpoint losses $L_{i} (λ_{2}) = 3 / 4 = α$ ; it is nonmonotone on type $a$ because $0 < 3 / 4$ as the grid index increases.

Counterexample

1.1A1A2F1

By [A1] and [F1], the two loss rows are exchangeable. By [A2], their values are bounded by $B = 1$ , the endpoint is safe, and the family violates the required nonincreasing-loss condition on an event of probability $1 / 10$ .

1.2A2F2algebra

If the calibration row is type $a$ , the first threshold is $(0 + 1) / 2 = 1 / 2 \leq 3 / 4$ , so the algorithm selects $λ_{1}$ . If it is type $b$ , the first threshold is $(1 + 1) / 2 = 1 > 3 / 4$ and the second is $(3 / 4 + 1) / 2 = 7 / 8 > 3 / 4$ , so the algorithm falls back to $λ_{2}$ .

2.1A1A2step 1.2algebra

Conditional on a type- $a$ calibration row, independence makes the expected test loss at $λ_{1}$ equal to $(1 / 10) 0 + (9 / 10) 1 = 9 / 10$ . Conditional on a type- $b$ calibration row, the selected endpoint loss is $3 / 4$ for either test type.

3.1A1A2step 2.1algebra∎

Averaging over the calibration row gives $E [L_{2} (\hat{λ})] = (1 / 10) (9 / 10) + (9 / 10) (3 / 4) = 153 / 200 = 3 / 4 + 3 / 200 > 3 / 4 = α$ . Thus the asserted guarantee fails despite exchangeability, boundedness, and a safe endpoint.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

7 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Anastasios N. Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster, Conformal Risk Control, arXiv:2208.02814v4 (standard reference, not scraped)
Anastasios N. Angelopoulos, Rina Foygel Barber, and Stephen Bates, Theoretical Foundations of Conformal Prediction, prepublication book manuscript (standard reference, not scraped)
Glenn Shafer and Vladimir Vovk, A Tutorial on Conformal Prediction, Journal of Machine Learning Research 9 (2008), 371–421 (standard reference, not scraped)