ExampleConstruction: AI-generatedVerification: AI-generatedPipeline-generatedprecheck pending

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

A two-token causal next-token mask

Example

For two scalar token states $x_{1} = 1$ and $x_{2} = 3$ , take query and key projections equal to zero, value projection equal to the identity, and the causal additive mask

M = (\begin{array}{cc} 0 & - \infty \\ 0 & 0 \end{array}) .

Row-softmax attention assigns the first query all weight to token $1$ and the second query equal weight to tokens $1$ and $2$ , so the output states are $(1, 2)$ .

Example references

Vaswani et al., §3.2.3: decoder masking of future positions

Dive into Deep Learning, §11.7.2: decoder self-attention with causal masking

Facts & Assumptions

Given: Two scalar token states, zero query and key projections, identity value projection, row softmax, and the causal mask in the Example.

[F1]

Masked attention replaces disallowed logits by $- \infty$ , requires at least one allowed key in each row, and normalizes each allowed row by softmax (Scaled dot-product attention).

[F2]

The decoder's causal mask permits key positions $j \leq i$ for query position $i$ (Positional encodings and causal masks).

Verification

1.1givenF1algebra

Both unmasked logits are zero. In the first row the mask leaves logits $(0, - \infty)$ , whose row-softmax is $(1, 0)$ ; multiplying by values $(1, 3)^{T}$ gives output $1$ .

2.1givenF1F2algebra∎

In the second row the mask leaves logits $(0, 0)$ , whose row-softmax is $(1 / 2, 1 / 2)$ ; multiplying by $(1, 3)^{T}$ gives output $2$ . Both rows have at least one allowed key, so the masked attention is defined.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

4 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Vaswani et al., Attention Is All You Need, §3.2.3 (standard reference, not scraped)
Dive into Deep Learning, §11.7.2 (standard reference, not scraped)