Pipeline-generated

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

✓ 4 results · all verified · 3 also independently AI-judged

Every result on this page is machine-checked by a proof checker and read in full and owner-audited; the judge is an additional, independent cross-model AI review of the proofs. The 1 not AI-judged were verified by owner audit (typically over a confirmed judge false positive), not failures.

Conformal Prediction and Exchangeable Ranks — Examples

1 · Prerequisites

2 · Summary

These examples compute a tied split quantile and a full-conformal candidate rank, then exhibit the limits of the guarantees. A rare subgroup can have poor conditional coverage even while marginal coverage exceeds its target, and a calibration/test distribution shift can produce zero coverage when exchangeability fails.

3 · Logical flowchart

4 · Definitions, theorems and proofs

None yet.

5 · Examples, counterexamples and false statements

ExampleConstruction: AI-generatedVerification: AI-generatedprecheck passOpen item page →

Split conformal coverage can be conservative when scores tie

Example

Take a constant score rule $s (z) = 0$ , two calibration examples, one exchangeable test example, and $α = 1 / 2$ . All three scores are tied. The ordinary split quantile set covers with probability $1$ , while the specified independent uniform tie-randomized rank set covers with probability exactly $1 / 2$ .

Facts & Assumptions

Given: $m = 2$ , $α = 1 / 2$ , all calibration and test scores equal $0$ , and an independent $U$ with Lebesgue probability law on $[0, 1]$ .

[F1]

The split algorithm sets $k = ⌈ (m + 1) (1 - α) ⌉$ , uses the $k$ th calibration score when $k \leq m$ , and otherwise uses $+ \infty$ . Split conformal prediction set

[F2]

The tie-randomized p-value counts strictly larger calibration scores and multiplies the full tie-group size, including the candidate, by $U$ . Conformal scores and rank p-values

[F3]

Under exchangeability, the randomized rank p-value is uniform. Conformal rank superuniformity under exchangeability

Verification

1.1givenF1

Here $k = ⌈ 3 / 2 ⌉ = 2 \leq m$ and the second calibration order statistic is $q = 0$ . The test score is also $0$ , so $0 \leq q$ and the nonrandomized prediction set always includes the true response.

1.2givenF2

There are no calibration scores strictly greater than the test score and there are two calibration ties; including the candidate gives tie count three. Thus $p_{U} = 3 U / 3 = U$ , and the tie-randomized set covers exactly when $U > 1 / 2$ , an event of probability $1 / 2$ .

2.1F3step 1.1step 1.2∎

The ordinary coverage $1$ is conservative relative to the nominal $1 - α = 1 / 2$ , whereas the randomized coverage is exact and agrees with [F3].

ExampleConstruction: AI-generatedVerification: AI-generatedprecheck passjudge pass (gpt-6-luna)Open item page →

Full conformal ranks a candidate in an augmented sample

Example

Take feature space $X = {x_{0}}$ and response space $Y = {1, 2, 3}$ , each with its power-set sigma-algebra, so the product example space is identified with $Y$ by $(x_{0}, y) \mapsto y$ . Let $Z_{1}, Z_{2}, Z_{3}$ be a uniformly random ordering of the pairs $(x_{0}, 1), (x_{0}, 2), (x_{0}, 3)$ , and use the response-coordinate score assignment $A_{i} (z_{1}, z_{2}, z_{3}) = y_{i}$ when $z_{i} = (x_{0}, y_{i})$ . The true candidate's nonrandomized upper-rank p-value is $1$ , $2 / 3$ , or $1 / 3$ , each with probability $1 / 3$ . At $α = 0.4$ , the full conformal set covers with probability $2 / 3$ .

Facts & Assumptions

Given: The six permutations of $((x_{0}, 1), (x_{0}, 2), (x_{0}, 3))$ have equal probability, and the score assignment returns each example's response coordinate.

[F1]

Exchangeability is invariance of the joint law under permutations. Exchangeable random elements

[F2]

The full score assignment is measurable and permutation-equivariant, and the prediction set includes the candidate when its upper-rank p-value exceeds $α$ . Full conformal prediction set

[F3]

The upper-rank p-value is superuniform for exchangeable scores. Conformal rank superuniformity under exchangeability

Verification

1.1givenF1

The six equally likely orderings are carried into one another by coordinate permutations, so the data tuple is exchangeable. Each response-coordinate map is measurable on the finite discrete product, and $A (τ_{π} z) = τ_{π} A (z)$ for every permutation.

1.2givenF2

If the last coordinate is $1$ , $2$ , or $3$ , respectively, the count of scores at least the candidate is $3$ , $2$ , or $1$ . Hence $p_{3}$ equals $1$ , $2 / 3$ , or $1 / 3$ , each with probability $1 / 3$ .

2.1F2F3step 1.2∎

Since $1 > 2 / 5$ and $2 / 3 > 2 / 5$ but $1 / 3 ≯ 2 / 5$ , the candidate is included in four of the six equally likely permutations, giving coverage $4 / 6 = 2 / 3 \geq 3 / 5$ . This is consistent with [F3].

CounterexampleConstruction: AI-generatedVerification: AI-generatedprecheck passjudge pass (gpt-6-luna)Open item page →

Marginal split coverage need not give rare-subgroup coverage

Statement refuted

Marginal split-conformal coverage at level $1 - α$ forces coverage at least $1 - α$ conditional on every feature subgroup.

Facts & Assumptions

Given: The feature group space ${A, B}$ and response space ${0, 1}$ have their power-set sigma-algebras. The group variable satisfies $P (G = A) = 0.01$ and $P (G = B) = 0.99$ , and the response is always $Y = 0$ . There are nine calibration examples and one test example, all mutually independent and identically distributed according to this law. The score is $s (G, y) = 1 {G = A}$ , and $α = 0.1$ .

[F1]

Independence of random elements means their generated sigma-algebras are independent, so events determined by distinct examples factor. Independent random elements

[F2]

The split algorithm uses $k = ⌈ (m + 1) (1 - α) ⌉$ and predicts $y$ when its score is at most the calibration threshold. Split conformal prediction set

[F3]

The split conformal theorem guarantees marginal coverage at least $1 - α$ under exchangeability. Split conformal marginal coverage

Counterexample

1.1givenF1F2

For $m = 9$ and $α = 1 / 10$ , $k = ⌈ 10 \cdot 9 / 10 ⌉ = 9$ , so $q$ is the maximum of the nine calibration scores. Each score is $1$ with probability $1 / 100$ and $0$ with probability $99 / 100$ . The events ${G_{i} = B}$ are measurable from their respective examples, so [F1] makes them mutually independent and independent of ${G_{10} = A}$ . Conditional on $G_{10} = A$ , no calibration example is in group $A$ with probability $\prod_{i = 1}^{9} P (G_{i} = B) = (99 / 100)^{9}$ . The test score is $1$ and is covered exactly when at least one calibration example has group $A$ ; thus the conditional coverage is $1 - (99 / 100)^{9} \approx 864827 / 10^{7} < 9 / 10$ .

2.1givenF2F3step 1.1∎

A group- $B$ test has score $0 \leq q$ and is always covered. Hence marginal coverage is $99 / 100 + (1 / 100) (1 - (99 / 100)^{9}) \approx 9908648 / 10^{7} \geq 9 / 10$ , consistent with [F3], while the rare-group conditional coverage violates the refuted claim.

CounterexampleConstruction: AI-generatedVerification: AI-generatedprecheck passjudge pass (gpt-6-luna)Open item page →

Exchangeability failure can destroy split coverage

Statement refuted

The split-conformal lower coverage bound holds for arbitrary calibration and test distributions, even when their joint law is not exchangeable.

Facts & Assumptions

Given: The feature is fixed at $x_{0}$ . The single calibration response is $Y_{1} = 0$ almost surely and the test response is $Y_{2} = 1$ almost surely. The training sigma-field is trivial, the response space is ${0, 1}$ , the score is $s (x, y) = y$ , $m = 1$ , and $α = 1 / 2$ .

[F1]

The split algorithm takes the $k$ th calibration score for $k = ⌈ (m + 1) (1 - α) ⌉ \leq m$ . Split conformal prediction set

[F2]

The split coverage theorem requires conditional exchangeability of calibration and test examples given training. Split conformal marginal coverage

Counterexample

1.1givenF2

The calibration and test examples are not exchangeable: the event that the first response is $0$ and the second is $1$ has probability $1$ , while the permuted event has probability $0$ .

2.1givenF1step 1.1∎

Here $k = ⌈ 2 (1 - 1 / 2) ⌉ = 1$ , so $q = Y_{1} = 0$ by [F1]. Thus $C (x_{0}) = {y \in {0, 1} : y \leq 0} = {0}$ , whereas $Y_{2} = 1$ almost surely. Coverage is $0$ , refuting the claim without exchangeability.