Support vector regression via gradient descent

The problem, from first principles

Linear SVM via gradient descent on hinge loss fits a classifier by minimizing hinge loss plus an L2 penalty. Support Vector Regression applies the same recipe to a real-valued target: swap hinge loss for the Epsilon-insensitive loss and keep the L2 penalty that keeps the weights small. Points inside the tube contribute no gradient, so only points outside it shape the fit.

Implement three functions with the exact signatures below. epsilon_insensitive_loss is imported from the previous question.

• svr_objective(weight, bias, X, y, epsilon, lambda_reg): mean epsilon-insensitive loss of X @ weight + bias against y, plus lambda_reg * ||weight||^2.
• svr_gradient(weight, bias, X, y, epsilon, lambda_reg): returns (grad_weight, grad_bias).
• train_linear_svr(X, y, lr=0.01, epochs=1000, epsilon=0.1, lambda_reg=0.0): plain gradient descent from zero weights and zero bias. Returns (weight, bias).

Constraints

• A residual inside the tube (|prediction - target| <= epsilon) contributes zero to the gradient.
• svr_gradient must match the true gradient of svr_objective away from the tube edges.
• Use the sign of the residual for points outside the tube.

Hints

Open one at a time. Each gives away a little more than the last.