Deep Double Descent: The Minimum-Norm Interpolator
Advanced
The problem, from first principles
Deep double descent (Nakkiran et al., 2019) shows test error can rise and then fall again as models grow past the point where they fit the training data exactly. Past that point, gradient descent from zero converges to the minimum-norm interpolating solution, which this question computes directly.
From theory to code
Implement min_norm_solution(X, y), returning the least-norm w with X w closest to y, using the pseudo-inverse.
Constraints
- Works when there are more features than samples, and when there are fewer.
Hints
Hint 1
np.linalg.pinv(X) @ y gives the minimum-norm least-squares solution.
Keywords
research-papersclassical-mllearning-theorydouble-descent
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.