AlgorithmAlgorithm: Literature-sourcedProof: Not applicablePipeline-generatedjudge pass (gpt-6-luna)

How statement and proof provenance work

The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.

Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.

These labels describe origin, not correctness: citations and verification chips remain separate evidence.

Finite-horizon model-predictive control

Definition

Input: a finite learned MDP model, a current state $s_{0}$ , a horizon $H \in N$ with $H \geq 1$ , and, for each state $s$ , an ordered finite list containing every action in $A (s)$ exactly once; this ordering determines deterministic tie-breaking. Initialize $V_{0} (s) = 0$ for every state. For $h = 1, \dots, H$ , compute for every admissible pair

$Q_{h} (s, a) = \hat{r} (s, a) + \sum_{s^{'} \in S} \hat{P} (s^{'} ∣ s, a) V_{h - 1} (s^{'}), V_{h} (s) = \max_{a \in A (s)} Q_{h} (s, a) .$

At the current state $s_{0}$ , return the first action in its supplied listing that attains $V_{H} (s_{0})$ . The controller executes only this first action in the true environment, observes the next state, and may invoke the procedure again with the new state and a newly supplied learned model. The procedure specifies no model-update rule and makes no guarantee about the true return of the model-selected action.

Definition references

A finite model provides an estimated reward and probability distribution for every admissible state-action pair.

Finite-horizon expected return is the sum of model-expected one-step rewards under state occupancies.

Every finite nonempty list of real action values has an attained maximum.

Levine, Kumar, Tucker, and Fu, Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems.

Depends on

Used by

Model exploitation under support shift Definition
A one-step optimizer selects a falsely rewarding unsupported action Example

Dependency tree · two levels

14 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.

Sources

Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu, Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (standard reference, not scraped)