sinusoidal-position-encoding — Amazon case

Amazon-inspired sequence model needs deterministic position information without adding learned position parameters. You need to construct the sinusoidal positional encoding for the requested sequence length and embedding dimension.

Build the sinusoidal positional encoding matrix of shape (n, d): for position $pp$ and column $cc$, with $r_c={10000}^{-2\lfloor c\mathrm{/}2\rfloor \mathrm{/}d}r\_c=10000^\{-2\backslash lfloor\; c/2\backslash rfloor/d\}$, even columns hold $\sin (p{r}_c)\backslash sin(p\backslash ,r\_c)$ and odd columns hold $\cos (p{r}_c)\backslash cos(p\backslash ,r\_c)$.

Implement solve(n,d).

Returns. Return a float NumPy array of shape (n, d).

Examples

Example 1

Input:

solve(2, 4)

Output:

[[0.0, 1.0, 0.0, 1.0], [0.841471, 0.540302, 0.01, 0.99995]]

Example 2

Input:

solve(3, 3)

Output:

[[0.0, 1.0, 0.0], [0.841471, 0.540302, 0.002154], [0.909297, -0.416147, 0.004309]]