Why OLS Requires More Observations Than Parameters
Problem statement
Why does standard inference in ordinary least squares require more observations than parameters? With $n$ observations and $p$ parameters, assess these claims: $n<p$ prevents a unique coefficient estimate and $n=p$ leaves no residual degrees of freedom, $n\le p$ means no least-squares solution exists, and $n\le p$ breaks standard errors, tests, and confidence intervals even if a fit is possible.
Show hint
How many independent pieces of information do the observations supply, compared with what the model has to pin down?
Relevant roles: Data Scientist, Applied Scientist and ML Engineer.