binary-cross-entropy — Mistral case

Mistral-inspired language-model evaluation component needs a stable binary cross-entropy calculation for a training diagnostic. You need to compute the loss from logits without introducing numerical overflow at extreme values.

Compute the mean binary cross-entropy from logits z and 0/1 labels y with a form that never overflows: $\max (z,0)-zy+\log (1+e^{-\mathrm{\mid }z\mathrm{\mid }})\backslash max(z,0)-zy+\backslash log(1+e^\{-|z|\})$ averaged over the samples.

Implement solve(z,y).

Returns. Return a non-negative Python float.

Examples

Example 1

Input:

solve([0.0, 2.0], [0, 1])

Output:

0.410038

Example 2

Input:

solve([1000.0, -1000.0], [1, 0])

Output:

0.0

Example 3

Input:

solve([1000.0], [0])

Output:

1000.0