Gradient of MSE with Respect to w and b
Intermediate classical-mllinear-regressionmanual-calculusgradients
Implement:
def mse_gradient(
input: np.ndarray,
weight: np.ndarray,
bias: np.ndarray | None,
target: np.ndarray,
) -> tuple[np.ndarray, np.ndarray | None]:
"""
input: shape (batch_size, in_features)
weight: shape (out_features, in_features)
bias: shape (out_features,), or None
target: shape (batch_size, out_features)
Returns:
grad_weight: same shape as weight
grad_bias: same shape as bias, or None if bias is None
"""
Use the same forward pass as 01-hypothesis-function's linear, mean-reduced over every element the way 02-mse-loss reduces.
Do not use an autograd library. These are the manual derivatives.