THE CHECKOUT GRADIENT

Difficulty: Medium Tags: Deep Learning, Binary Cross-Entropy, Sigmoid, Backpropagation

Story

A commerce ranking model predicts whether a product impression leads to a purchase. The training service needs the gradient of binary cross-entropy with respect to each logit, not with respect to the post-sigmoid probability. A common implementation bug applies the sigmoid derivative twice.

The Math

For logit z_i and label y_i in {0,1}, with p_i = sigmoid(z_i):

L = -(1/n) * sum_i [y_i*log(p_i) + (1-y_i)*log(1-p_i)]
dL/dz_i = (p_i - y_i) / n

Use numerically stable calculations for the loss.

Input Format

logits (list of floats), labels (list of 0/1 ints).

Output Format

(mean_loss, gradients) (6 decimal places each).

Constraints

• 1 <= n <= 10^5, -30 <= z_i <= 30, y_i in {0,1}.
• Time limit: 1.0 second.

Example

Input

logits = [0, 2], labels = [0, 1]

Output

mean_loss = 0.410038
gradients = [0.250000, -0.059601]

Explanation: p_1=0.5, p_2≈0.880797. The mean loss is (log 2 + log(1+e^-2))/2 ≈ 0.410038. The gradients are (0.5-0)/2=0.25 and (0.880797-1)/2≈-0.059601.