Momentum Update

Perform one SGD-with-momentum update using the (PyTorch-style) rule $v\leftarrow \mu v+gv\backslash leftarrow\backslash mu\; v+g$, then $w\leftarrow w-\eta vw\backslash leftarrow\; w-\backslash eta\; v$.

Implement solve(w, v, grad, lr, mu).

Returns. Return a tuple (new_velocity, new_weights).

Examples

Example 1

Input:

solve([1.0], [0.0], [2.0], 0.1, 0.9)

Output:

([2.0], [0.8])

Example 2

Input:

solve([0.92], [2.0], [-1.0], 0.1, 0.9)

Output:

([0.8], [0.84])