Forward selection with a model in the loop

Let the model decide which feature to add next

A wrapper method judges a feature set by actually fitting a model on it. Forward selection starts from nothing and, at each step, adds the single feature that gives the best fit when combined with the features already chosen.

Implement forward_selection(X, y, k). It returns the list of k selected column indices, in the order they were picked.

At each step, for every feature not yet chosen:

1. Fit ordinary least squares with an intercept on the chosen columns plus this one.
2. Score the fit by R^2 = 1 - SS_res / SS_tot. If SS_tot is zero, the score is 0.0.

Pick the feature with the highest score. Ties go to the smallest index.

Constraints

• X is 2-D with d columns, y has one value per row.
• 1 <= k <= d, otherwise raise ValueError.
• Do not modify the inputs.

Hints