Why OLS Requires More Observations Than Parameters

Problem statement

Why does standard inference in ordinary least squares require more observations than parameters? With $n$ observations and $p$ parameters, assess these claims: $n<p$ prevents a unique coefficient estimate and $n=p$ leaves no residual degrees of freedom, $n\le p$ means no least-squares solution exists, and $n\le p$ breaks standard errors, tests, and confidence intervals even if a fit is possible.

Show hint

How many independent pieces of information do the observations supply, compared with what the model has to pin down?

Relevant roles: Data Scientist, Applied Scientist and ML Engineer.

Related free problems