Gradient Clipping by Norm

Clip a list of gradient arrays by their global norm. Compute $\mathrm{\parallel }g\mathrm{\parallel }=\sqrt{{\sum }_k\mathrm{\parallel }{g}_k{\mathrm{\parallel }}^2}\backslash |g\backslash |=\backslash sqrt\{\backslash sum\_k\backslash |g\_k\backslash |^2\}$ over all arrays together; if it exceeds clip, multiply every array by clip / ||g||, otherwise leave them unchanged.

Implement solve(grads,clip).

Returns. Return a list of NumPy arrays with the same shapes as the inputs. If the global norm is $00$ the arrays are returned unchanged.

Examples

Example 1

Input:

solve([[3.0, 4.0]], 1.0)

Output:

[[0.6, 0.8]]

Example 2

Input:

solve([[3.0], [4.0]], 10.0)

Output:

[[3.0], [4.0]]

Example 3

Input:

solve([[3.0], [4.0]], 2.5)

Output:

[[1.5], [2.0]]