Permutation Feature Importance

EasyML Coding

Problem statement

Implement Permutation Feature Importance (PFI) for a linear regression model. PFI measures how much the model's score degrades when the values of a single feature are randomly shuffled, breaking the relationship between that feature and the target.

For reproducibility, instead of generating random permutations internally, your function will receive explicit permutation index arrays.

Inputs
  • X: 2D list/array of shape (n_samples, n_features)
  • y: 1D list/array of shape (n_samples,)
  • weights: 1D list/array of model coefficients of shape (n_features,)
  • bias: scalar intercept term (float)
  • permutations: a list of length n_features. Element j is itself a list of one or more permutation index arrays (each a permutation of [0, 1, ..., n_samples-1]). These are the shuffles to apply to column j.
Model

The model predicts $\hat{y} = X w + b$.

Score

Use the coefficient of determination $R^2$: $$R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2}$$

Output

Return a list of length n_features. Entry j is the importance of feature j, defined as the baseline $R^2$ minus the mean $R^2$ over all permutation repeats applied to column j (other columns left untouched). A larger value indicates a more important feature.

Do not round inside the solution.

Example

Input:

X = [[1,0],[2,0],[3,1],[4,1]], y = [1,2,3,4], weights = [1,0], bias = 0, permutations = [[[3,2,1,0]], [[1,0,3,2]]]

Output:

[4.0, 0.0]

Reasoning: With weights=[1,0], predictions equal X[:,0], so baseline R^2=1.0 (perfect fit). Permuting column 0 with [3,2,1,0] reverses it to [4,3,2,1]; predictions become [4,3,2,1], yielding SS_res=20 and SS_tot=5, so R^2=-3. Importance of feature 0 = 1.0 - (-3.0) = 4.0. Feature 1 has weight 0, so any permutation leaves predictions unchanged: importance = 0.0.

Show hint

What happens to a model's score if one input column is scrambled so that it no longer lines up with the target, while its values stay the same?

Relevant roles: Data Scientist, ML Engineer, Applied Scientist and Research Scientist.

Related free problems