Mixed Precision Loss Scale

Apply static loss scaling for mixed-precision training. Multiply the loss by scale, and divide every gradient array in scaled_grads by the same scale to recover the true gradients. scale must be positive.

Implement solve(loss, scaled_grads, scale).

Returns. Return (scaled_loss, unscaled_grads) where the second item is a list of NumPy arrays; a non-positive scale raises ValueError.

Examples

Example 1

Input:

solve(2.0, [[8.0, 4.0]], 4.0)

Output:

(8.0, [[2.0, 1.0]])

Example 2

Input:

solve(0.5, [[-6.0]], 2.0)

Output:

(1.0, [[-3.0]])