Bias-variance tradeoff

The problem, from first principles

02-classification-metrics measures how wrong a model's predictions are on average. It doesn't explain why a model is wrong. A model too simple to represent the true pattern will be wrong the same way on every training set it's given — that's bias. A model flexible enough to chase the specific noise in whatever training set it happened to see will be wrong differently every time — that's variance. Two models can have the same average test error for completely different reasons, and knowing which reason applies is what tells you whether to make the model more flexible or less.

This is the same tension every model in this curriculum faces implicitly, from a straight-line fit to a deep tree: too rigid and it can't capture the pattern regardless of data; too flexible and it fits each dataset's particular noise instead of the pattern underneath.

From theory to code

Implement polynomial_features, fit_polynomial, and predict_polynomial to produce many independently-trained polynomial fits of varying complexity, then bias_variance_decomposition(predictions, targets) to split their collective test error into the two pieces Theory separates.

Constraints

• polynomial_features(x, degree): returns shape (n_samples, degree+1), columns 1, x, x^2, ..., x^degree.
• fit_polynomial(x, y, degree): least-squares fit, returns coefficients of shape (degree+1,).
• predict_polynomial(coefficients, x): evaluates a fitted polynomial at new x values; degree is inferred from coefficients, not passed separately.
• bias_variance_decomposition(predictions, targets): predictions has shape (n_models, n_test_points), one row per independently-trained model's predictions on the same test points; targets has shape (n_test_points,), the true, noiseless function value at each point (not a noisy training label).
• Returns (bias_squared, variance, bias_squared + variance), each averaged over test points.
• variance measures disagreement across models at each test point, not across test points for one model — the two axes are not interchangeable.

Hints

Open one at a time. Each gives away a little more than the last.