Chapter 1: Classical ML

Prediction intervals and conformal prediction

You will be able to distinguish ranges for averages from ranges for individuals. You will build a conformal interval and state its guarantee.

18 minInteractive lessonChapter 1: Classical MLLesson 17 of 21

What you will learn

  • Tell a confidence interval from a prediction interval and say why the second is wider
  • Build a split conformal interval from calibration scores
  • Say what the guarantee needs (exchangeable data)

Key terms

Noise
Deviation from the average.
Standard deviation
Spread in original units.
Normal noise
Bell-shaped noise.
Density
A curve whose area describes fractions of outcomes.

Prerequisites

Take Calibration and class imbalance first. It comes right before this lesson in the course.

Where this lesson picks up: You checked whether probabilities matched outcomes. What range can describe one future outcome?

What comes next

How can you explain why a model made its prediction?

Next lesson: Explaining a model: permutation importance and SHAP

More lessons in this part

Take the lesson

Short cards, one idea at a time, with quick checks as you go.