Chapter 1: Classical ML
Prediction intervals and conformal prediction
You will be able to distinguish ranges for averages from ranges for individuals. You will build a conformal interval and state its guarantee.
What you will learn
- Tell a confidence interval from a prediction interval and say why the second is wider
- Build a split conformal interval from calibration scores
- Say what the guarantee needs (exchangeable data)
Key terms
- Noise
- Deviation from the average.
- Standard deviation
- Spread in original units.
- Normal noise
- Bell-shaped noise.
- Density
- A curve whose area describes fractions of outcomes.
Prerequisites
Take Calibration and class imbalance first. It comes right before this lesson in the course.
Where this lesson picks up: You checked whether probabilities matched outcomes. What range can describe one future outcome?
What comes next
How can you explain why a model made its prediction?
Next lesson: Explaining a model: permutation importance and SHAP
More lessons in this part
- ROC, PR curves and AUC 15 min
- Calibration and class imbalance 20 min
- Explaining a model: permutation importance and SHAP 17 min
Take the lesson
Short cards, one idea at a time, with quick checks as you go.