layernorm-row — Atlassian case

Atlassian-inspired sequence model normalizes each token representation independently before passing it to the next block. You need to implement row-wise LayerNorm with the supplied scale and bias parameters.

Apply layer normalisation to every row of X (the last axis): subtract the row mean, divide by $\sqrt{\text{row variance}+\epsilon }\backslash sqrt\{\backslash text\{row\; variance\}+\backslash varepsilon\}$ (population variance, eps defaults to ${10}^{-5}10^\{-5\}$), then scale by gamma and shift by beta. gamma and beta may be scalars or arrays that broadcast over the last axis.

Implement solve(X,gamma,beta,eps=1e-5).

Returns. Return a float NumPy array with the shape of X.

Examples

Example 1

Input:

solve([[1, 2, 3]], 1, 0)

Output:

[[-1.224736, 0.0, 1.224736]]

Example 2

Input:

solve([[1.0, 3.0], [10.0, 30.0]], [1.0, 2.0], [0.0, 1.0], eps=0.0)

Output:

[[-1.0, 3.0], [-1.0, 3.0]]