Underfit or Overfit: Degree-4 vs Degree-6 Fit to a Quintic
Problem statement
Which polynomial predicts better, and why? Data follow $y=f(x)+\epsilon$, with $\epsilon\sim N(0,1)$ and $x$ uniform on $[-3,3]$. The true polynomial has degree 5, an $x^5$ coefficient of 0.5, and other coefficients of order 1. Using 1000 training pairs and 100 separate test pairs, compare ordinary least squares models of degree 4 (A) and degree 6 (B).
Show hint
Besides the irreducible noise, what two kinds of error does a fitted model add on new data?
Relevant roles: ML Engineer, Applied Scientist and Data Scientist.