AlgorithmAlgorithm: Literature-sourcedProof: Not applicablePipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Holdout, leave-out, V-fold, and repeated cross-validation

Definition

Input: observations $z_{1}, \dots, z_{n}$ , learning rules $A_{k}$ , a loss $L$ , and a finite split schedule $((T_{r}, V_{r}, w_{r}))_{r = 1}^{B}$ . Each $(T_{r}, V_{r})$ is a split as in Cross-validation splits and training-size risk targets, $w_{r} \geq 0$ , and $\sum_{r = 1}^{B} w_{r} = 1$ . A randomized schedule is admissible only when its law is specified and is independent of the observations.

Output: the cross-validation estimate

$\hat{R} : = \sum_{r = 1}^{B} w_{r} \frac{1}{∣ V_{r} ∣} \sum_{i \in V_{r}} L (A_{∣ T_{r} ∣} (z_{T_{r}}), z_{i}),$

with each training tuple ordered by increasing index.

Instructions: fit the rule on each training set, evaluate its loss on every observation in that split's validation set, average within that validation set, then combine the split averages using the declared weights. To compare two or more candidate rules, apply the same schedule and subtract their validation losses within each split before averaging; this is the paired comparison.

The following choices give standard procedures:

Holdout: one split, with $T$ the chosen training set, $V = T^{c}$ , and weight $1$ .
Leave-one-out: for $n \geq 2$ , use $n$ splits $T_{i} = [n] ∖ {i}$ , $V_{i} = {i}$ , with weights $1 / n$ .
Leave- $p$ -out: for $n \geq 2$ and $1 \leq p < n$ , use one split for every $p$ -element subset $J \subseteq [n]$ , with $T_{J} = J^{c}$ , $V_{J} = J$ , and weight $1 / (\binom{n}{p})$ .
$K$ -fold: for $2 \leq K \leq n$ and a partition $I_{1}, \dots, I_{K}$ into nonempty folds, use $T_{j} = I_{j}^{c}$ , $V_{j} = I_{j}$ , and $w_{j} = ∣ I_{j} ∣ / n$ ; for equal folds this is $1 / K$ .
Repeated $K$ -fold: for $n \geq 2$ , $2 \leq K \leq n$ and $R \geq 1$ , apply the $K$ -fold schedule to each of $R$ declared partitions and give fold $I_{r j}$ weight $∣ I_{r j} ∣ / (R n)$ . If partitions are randomized, record their independent randomization law and average under that law or over the declared finite repetitions.

These instructions define estimates only. Their validity as estimates of a deployment target depends on matching the split, sampling unit, training size, and averaging law to that target.

Depends on

Cross-validation splits and training-size risk targets

Used by

Dependency tree · two levels

5 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Sylvain Arlot and Alain Celisse, A survey of cross-validation procedures for model selection, §4 (standard reference, not scraped)
Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani, An Introduction to Statistical Learning, 7th printing, Chapter 5 §5.1 (standard reference, not scraped)