Permutation Feature Importance
Problem statement
Implement Permutation Feature Importance (PFI) for a linear regression model. PFI measures how much the model's score degrades when the values of a single feature are randomly shuffled, breaking the relationship between that feature and the target.
For reproducibility, instead of generating random permutations internally, your function will receive explicit permutation index arrays.
Inputs
X: 2D list/array of shape(n_samples, n_features)y: 1D list/array of shape(n_samples,)weights: 1D list/array of model coefficients of shape(n_features,)bias: scalar intercept term (float)permutations: a list of lengthn_features. Elementjis itself a list of one or more permutation index arrays (each a permutation of[0, 1, ..., n_samples-1]). These are the shuffles to apply to columnj.
Model
The model predicts $\hat{y} = X w + b$.
Score
Use the coefficient of determination $R^2$: $$R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2}$$
Output
Return a list of length n_features. Entry j is the importance of feature j, defined as the baseline $R^2$ minus the mean $R^2$ over all permutation repeats applied to column j (other columns left untouched). A larger value indicates a more important feature.
Do not round inside the solution.
Example
Input:
X = [[1,0],[2,0],[3,1],[4,1]], y = [1,2,3,4], weights = [1,0], bias = 0, permutations = [[[3,2,1,0]], [[1,0,3,2]]]
Output:
[4.0, 0.0]
Reasoning: With weights=[1,0], predictions equal X[:,0], so baseline R^2=1.0 (perfect fit). Permuting column 0 with [3,2,1,0] reverses it to [4,3,2,1]; predictions become [4,3,2,1], yielding SS_res=20 and SS_tot=5, so R^2=-3. Importance of feature 0 = 1.0 - (-3.0) = 4.0. Feature 1 has weight 0, so any permutation leaves predictions unchanged: importance = 0.0.
Show hint
What happens to a model's score if one input column is scrambled so that it no longer lines up with the target, while its values stay the same?
Relevant roles: Data Scientist, ML Engineer, Applied Scientist and Research Scientist.
Related free problems
- Multi-Head AttentionMedium
- Masked Self-AttentionMedium