How OLS Estimates Move Under Shift, Scale, and Duplication
Problem statement
Consider ordinary least squares regression (both the simple and multiple-regression cases behave the same way here). Take a fitted regression and apply each of the following transformations separately, then say what happens to the estimates:
- Shift: add the same constant (say 2) to all the data values.
- Scale: multiply all the data values by the same constant (say 2).
- Duplicate: copy every data point once, doubling the sample size (the new dataset is the original stacked on top of itself).
For each scenario, state how these change (or do not change): the slope coefficient $\hat{\beta}$, the intercept, the standard errors, the t-statistics, $R^2$, and adjusted $R^2$. Assume "all the data values" means every variable ($y$ and every regressor) is transformed the same way.
Show hint
Which summary sums does every OLS output (slope, intercept, residual variance, degrees of freedom) actually depend on?
Relevant roles: Data Scientist, Applied Scientist and ML Engineer.