ExampleConstruction: AI-generatedVerification: AI-generatedPipeline-generatedprecheck pendingjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

An inaccurate model can give the exact return on behavior support

Example

Let $S = {s, g}$ , with one dummy action $d$ at $g$ and actions $a, b$ at $s$ . At $g$ , action $d$ has reward $0$ and returns to $g$ . At $s$ , action $a$ has reward $1$ and moves to $g$ , while action $b$ returns to $s$ with true reward $0$ and learned reward $1$ . The true and learned models agree on $(s, a)$ and $(g, d)$ . Let the supported set be $B = {(s, a), (g, d)}$ , let the initial state be $s$ , and let $H \geq 2$ . A policy that chooses $a$ at $s$ and $d$ at $g$ has identical true and learned $H$ -step returns, although the learned model is wrong on the unsupported pair $(s, b)$ .

The TikZ diagram shows the two model transition/reward tables: a label $u : r$ gives action $u$ and its immediate reward on the arrow, and an arrow returning to its source denotes a self-transition. The learned reward for $b$ is marked with a hat.

\begin{center} \begin{tikzpicture}[>=stealth, node distance=32mm] \node[circle,draw] (s) { $s$ }; \node[circle,draw,right of=s] (g) { $g$ }; \draw[->] (s) to[bend left=12] node[above] { $a : 1$ } (g); \draw[->] (g) to[loop right] node[right] { $d : 0$ } (g); \draw[->] (s) to[loop left] node[left] { $b : 0 (\hat{b} : 1)$ } (s); \end{tikzpicture} \end{center}

Example references

The true and learned reward and transition tables share a finite state-action domain, and errors can be restricted to a pair set.

A fixed policy's finite-horizon return difference is bounded by reachable-pair reward and transition errors.

Facts & Assumptions

Given: The two-state models, supported set, initial law, horizon, and policy in the Example.

[F1]

Finite learned transition and reward models: the true and learned reward means lie in $[0, 1]$ , and each transition row is a probability distribution.

[F2]

Finite-horizon simulation bound for one fixed policy: for a common initial law and fixed Markov policy, uniform errors on the union of reachable pairs give the stated finite-horizon return bound.

Verification

Given: The finite models and policy in the Example.

1.1givenF1F2algebra

The true and learned occupancies under the policy place mass only on $(s, a)$ at time $0$ and on $(g, d)$ at each later time. These pairs are exactly the listed supported pairs, and the models agree on both, so the reachable reward and transition errors are zero. Theorem [F2] applies with $ϵ_{r} = ϵ_{p} = 0$ .

2.1step 1.1F2algebra∎

Substitution into [F2] gives $∣ J_{H} - {\hat{J}}_{H} ∣ \leq 0$ . Since an absolute value is nonnegative, $J_{H} = {\hat{J}}_{H}$ . Directly, each return is $1$ from the reward at the first step and $0$ thereafter. Thus model errors outside the fixed policy's support do not affect this return.

Depends on

Used by

Nothing in the library uses this result yet.

Dependency tree · two levels

12 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.