Two-Layer MLP Backward

Backpropagate through the two-layer ReLU network of the previous problem. Given the input X, the upstream gradient dY of the loss with respect to the output, the weights W1, W2 and the forward cache (z1, h), compute the gradients of every parameter and of the input.

Implement solve(X, dY, W1, W2, cache).

Returns. Return a tuple (dX, dW1, db1, dW2, db2) of NumPy arrays with the shapes of X, W1, b1, W2 and b2.

Examples

Example 1

Input:

solve([[1.0, 2.0]], [[1.0]], [[1.0, 0.0], [0.0, 1.0]], [[2.0], [3.0]], ([[0.0, 3.0]], [[0.0, 3.0]]))

Output:

([[0.0, 3.0]], [[0.0, 3.0], [0.0, 6.0]], [0.0, 3.0], [[0.0], [3.0]], [1.0])

Example 2

Input:

solve([[2.0]], [[1.0]], [[1.0]], [[4.0]], ([[1.0]], [[1.0]]))

Output:

([[4.0]], [[8.0]], [4.0], [[1.0]], [1.0])