Alphabeta MathHow statement and proof provenance work
The first chip identifies the source of the statement or construction; the second identifies the source of its local proof or verification.
- Literature-sourced: the exact statement appears in a cited source; only wording and notation differ.
- AI-adapted: a semantically identical restatement of literature-sourced material, modulo indexing, notation, and boundary cases adopted by the library.
- AI-generated: a genuinely novel statement formulated by AI, with no source for the claim itself.
These labels describe origin, not correctness: citations and verification chips remain separate evidence.
Bellman optimality operator
Definition
For a finite MDP and a discount , define the Bellman optimality operator by
The maximum exists because each is finite and nonempty. If , this is the unique map from to itself.
Definition references
The finite state-dependent action sets, expected reward, and transition probabilities.
A maximum is an attained upper bound belonging to the set.
Every finite nonempty set of real action values has an attained maximum.
Sutton and Barto, Reinforcement Learning: An Introduction.
Kaelbling, Littman, and Cassandra, Brown Technical Report CS-96-08.
Depends on
Used by
- Exact finite policy iteration with retained greedy actions Algorithm
- Exact value iteration for the Bellman optimality operator Algorithm
- A common reward shift moves every one-state Bellman value equally Example
- A one-state value-iteration model attains the Bellman error bound Example
- Changing a tied greedy action can cycle, while retention stops Example
- The optimal action value is attained by a greedy stationary policy Lemma
- Finite-MDP Bellman operators are gamma contractions with unique fixed points Theorem
- Greedy policy improvement and finite policy-iteration termination Theorem
- Tabular Q-learning converges under per-pair Robbins–Monro steps and full visitation Theorem
Dependency tree · two levels
16 results within two dependency steps of this one, each drawn at its shortest distance from it. An arrow runs from a result to what uses it, so the chart reads left to right and ends at this result, which carries a heavier outline. Every node is a link to that result. Click elsewhere on the chart to enlarge it.
Sources
- Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd ed. (standard reference, not scraped)
- Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra, Planning and Acting in Partially Observable Stochastic Domains, Brown CS-96-08 (standard reference, not scraped)