Chapter 5: Reinforcement learning and post-training

Value iteration and the greedy policy

A robot must compare rewards now with rewards later. Interviews ask how that comparison becomes a policy.

18 minInteractive lessonChapter 5: Reinforcement learning and post-trainingLesson 2 of 13

What you will learn

  • Run value iteration and read the greedy policy off the values

Key terms

sweep
one pass that gives every room a new value from the old values
greedy policy
in each room, take the move with the largest score

Prerequisites

Take MDPs: discounted returns and the Bellman expectation equation first. It comes right before this lesson in the course.

Where this lesson picks up: You found values for a fixed policy. Which policy gives the best return?

What comes next

Next: what if the robot has no map of rewards and moves, and must learn from experience instead?

Next lesson: Learning from experience: TD error, Q-learning and SARSA

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.