Chapter 5: Reinforcement learning and post-training
Value iteration and the greedy policy
A robot must compare rewards now with rewards later. Interviews ask how that comparison becomes a policy.
What you will learn
- Run value iteration and read the greedy policy off the values
Key terms
- sweep
- one pass that gives every room a new value from the old values
- greedy policy
- in each room, take the move with the largest score
Prerequisites
Take MDPs: discounted returns and the Bellman expectation equation first. It comes right before this lesson in the course.
Where this lesson picks up: You found values for a fixed policy. Which policy gives the best return?
What comes next
Next: what if the robot has no map of rewards and moves, and must learn from experience instead?
Next lesson: Learning from experience: TD error, Q-learning and SARSA
More lessons in this part
- MDPs: discounted returns and the Bellman expectation equation 15 min
- Learning from experience: TD error, Q-learning and SARSA 18 min
- Bandits: regret, epsilon-greedy and UCB 16 min
- Thompson sampling: picking by sampling your beliefs 14 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.