Transformer Residual Block

Apply a residual connection around a sublayer: return $x+\text{sublayer}(x)x+\backslash text\{sublayer\}(x)$. sublayer is a function (for example attention or a feed-forward network) that returns an array of the same shape as x.

Implement solve(x, sublayer).

Returns. Return a NumPy array with the shape of x.

Examples

Example 1

Input:

solve([1.0, 2.0], lambda v: v * 2)

Output:

[3.0, 6.0]

Example 2

Input:

solve([[1.0, -1.0]], lambda v: np.zeros_like(v))

Output:

[[1.0, -1.0]]