Choosing K in KNN and the Bias-Variance Tradeoff

EasyClassical ML

Reported in interviews at Deloitte The Voleon Group

Problem statement

  1. In $k$-nearest-neighbors, how do you choose the value of $k$? Explain how $k$ controls the bias-variance tradeoff.
  1. Relatedly, in $k$-fold cross-validation, how does the choice of the number of folds $k$ affect the bias and variance of your estimate of test error?
Show hint

What happens at the two extremes of the neighbor count, one neighbor versus every training point?

Relevant roles: Data Scientist, Applied Scientist and ML Engineer.

Related free problems