Alphabeta MathDefinitionDefinition: Literature-sourcedProof: Not applicablePipeline-generatedjudge pass (gpt-6-luna)
How statement and proof provenance work
The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.
- Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
- AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
- AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.
These labels describe origin, not correctness: citations and verification chips remain separate evidence.
Policy Bellman operator
Definition
For a finite MDP, a discount , and a stationary Markov policy , define the policy Bellman operator by
If , this is the unique map from to itself.
Depends on
Used by
- Exact finite policy iteration with retained greedy actions Algorithm
- Tabular TD(0) policy prediction Algorithm
- State-weighted Bellman error and projected Bellman error Definition
- A common reward shift moves every one-state Bellman value equally Example
- Changing a tied greedy action can cycle, while retention stops Example
- Expected discounted policy value is a fixed point of its Bellman operator Lemma
- The optimal action value is attained by a greedy stationary policy Lemma
- The weighted feature projection is well-defined Lemma
- Expected off-policy TD(0) diverges in Baird’s seven-state example Theorem
- Feature-reward data do not determine the Bellman error or, at positive discount, its minimizer Theorem
- Finite-MDP Bellman operators are gamma contractions with unique fixed points Theorem
- Greedy policy improvement and finite policy-iteration termination Theorem
Dependency tree · two levels
12 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.
Sources
- Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd ed. (standard reference, not scraped)
- Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra, Planning and Acting in Partially Observable Stochastic Domains, Brown CS-96-08 (standard reference, not scraped)