ExampleConstruction: AI-generatedVerification: AI-generatedPipeline-generatedprecheck pendingjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

A one-step optimizer selects a falsely rewarding unsupported action

Example

There is one state $s$ , two actions $a_{s a f e}$ and $a_{o f f}$ , and both actions return deterministically to $s$ . Their true rewards are $1 / 2$ and $0$ , respectively; their learned rewards are $1 / 2$ and $1$ . Let $H = 1$ , let $B = {(s, a_{s a f e})}$ , and let the finite policy class contain the two policies that choose one action at time $0$ . Supply MPC with the ordered action listing $(a_{s a f e}, a_{o f f})$ at $s$ . Finite-horizon model-predictive control chooses $a_{o f f}$ , whose predicted return is $1$ and true return is $0$ , while the true best return in the class is $1 / 2$ .

The diagram uses a vertex for the state and a loop for each deterministic self-transition; each loop label gives the action and its true/learned immediate rewards.

\begin{center} \begin{tikzpicture}[>=stealth] \node[circle,draw] (s) at (0,0) { $s$ }; \draw[->] (s) to[loop above] node[above] { $a_{s a f e} : 1 / 2, \hat{r} = 1 / 2$ } (s); \draw[->] (s) to[loop below] node[below] { $a_{o f f} : 0, \hat{r} = 1$ } (s); \end{tikzpicture} \end{center}

Example references

Off-support exploitation requires learned occupancy outside the data-supported set, overprediction, and true suboptimality.

Finite-horizon MPC returns a first action maximizing the learned-model finite-horizon value.

Facts & Assumptions

Given: The one-state model pair, data set, horizon, and policy class in the Example.

[F1]

Model exploitation under support shift: a policy is model-optimal if it attains the learned return maximum in the finite policy class; off-support exploitation additionally requires positive learned occupancy on an unsupported pair, overprediction, and true suboptimality.

[F2]

Finite-horizon model-predictive control: at horizon one, the returned action maximizes the learned immediate reward, with the complete supplied action listing used only to break ties.

Verification

Given: The finite model pair and class in the Example.

1.1givenF2algebra

The two horizon-one learned returns are $1 / 2$ and $1$ , so $a_{o f f}$ is the unique learned optimizer and [F2] returns it. Its learned occupancy on $(s, a_{o f f})$ is $1$ , although this pair is outside $B$ .

2.1givenF1step 1.1algebra∎

The true returns of $a_{s a f e}$ and $a_{o f f}$ are $1 / 2$ and $0$ . Thus the learned optimizer overpredicts its own return ( $1 > 0$ ) and is truly suboptimal ( $0 < 1 / 2$ ). All three clauses of [F1] hold, so this is off-support model exploitation as defined.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

5 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.