Two-Layer MLP Forward

Compute the forward pass of a two-layer network with a ReLU hidden layer: $z_1=X{W}_1+b_{1}z\_1=XW\_1+b\_1$, $h=\max (z_1,0)h=\backslash max(z\_1,0)$, $Y=h{W}_2+b_{2}Y=hW\_2+b\_2$. Also return the intermediate values needed for backpropagation.

Implement solve(X, W1, b1, W2, b2).

Returns. Return a tuple (Y, (z1, h)) where the second element is the cache of pre-activation and hidden activation.

Examples

Example 1

Input:

solve([[1, 2]], [[1, 0], [0, 1]], [-1, 1], [[2], [3]], [0])

Output:

([[9]], ([[0, 3]], [[0, 3]]))

Example 2

Input:

solve([[2]], [[1]], [-1], [[4]], [1])

Output:

([[5]], ([[1]], [[1]]))