ExampleConstruction: AI-generatedVerification: AI-generatedPipeline-generatedprecheck passjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

A different dropout mask changes the reverse result

Example

Consider the optional inverted-dropout extension with dropout rate $ρ = 1 / 2$ , in which a fixed realization $ξ_{j} \in {0, 1}$ multiplies attention weight $p_{j}$ by $ξ_{j} / (1 - ρ)$ . Take $n = 2$ , $d = v = 1$ , $Q = K = [\begin{array}{c} 0 \\ 0 \end{array}]$ , $V = [\begin{array}{c} 2 \\ 0 \end{array}]$ , $c = 1$ , and let both query rows allow both keys. For upstream matrix $G = [\begin{array}{c} 1 \\ 0 \end{array}]$ , the forward realization $ξ = (1, 0)$ has row- $1$ output $2$ and full value gradient $(d V_{1}, d V_{2}) = (1, 0)$ . Recomputing with the different realization $ξ^{'} = (0, 1)$ would give value gradient $(0, 1)$ for the same upstream matrix, so it is not the reverse derivative of the forward realization.

Example references

The base procedure and its same-input recomputation convention

Facts & Assumptions

Given: The optional dropout rule, the fixed inputs, and the two dropout realizations stated in the example.

[F1]

Exact tiled softmax attention with block-recomputed vector-Jacobian products: the base procedure has no dropout; its reverse pass uses the same fixed attention mask and recomputed scores.

Verification

1.1F1givenalgebra

All scores are zero, so the undropped probabilities are $(1 / 2, 1 / 2)$ . Under $ξ = (1, 0)$ and $ρ = 1 / 2$ , the dropped weights are $(1, 0)$ , hence query row $1$ has output $2$ and derivative with respect to $(V_{1}, V_{2})$ equal to $(1, 0)$ . Since $G_{2} = 0$ , query row $2$ contributes zero to the full value gradient, which is therefore $(1, 0)$ .

2.1F1step 1.1algebra∎

Under $ξ^{'} = (0, 1)$ the dropped weights are $(0, 1)$ , so the recomputed value derivative is $(0, 1)$ . This differs from the derivative of the fixed forward realization in step 1.1, confirming that this extension must replay $ξ$ .

Depends on

Exact tiled softmax attention with block-recomputed vector-Jacobian products

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

4 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Tri Dao et al., FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness, Appendix B.2–B.4 (standard reference, not scraped)