Gradient of BCE
Intermediate classical-mlclassificationmanual-calculusgradients
Implement:
def bce_gradient(
input: np.ndarray,
p: np.ndarray,
target: np.ndarray,
) -> tuple[np.ndarray, np.ndarray]:
"""
input: shape (batch_size, in_features)
p: shape (batch_size, 1), sigmoid(linear(input, weight, bias))
target: shape (batch_size, 1)
Returns:
grad_weight: shape (1, in_features)
grad_bias: shape (1,)
"""
p/target follow 01-hypothesis-function's convention, (batch_size, 1), never squeezed to (batch_size,), so grad_weight/grad_bias come out matching weight's (1, in_features) / bias's (1,) shape directly, the same convention Linear Regression's 03-mse-gradient uses.