Bahdanau Attention: Additive Alignment Scores
Intermediate
The problem, from first principles
Bahdanau et al. (2014) let a decoder look back at every encoder state when producing each output word. The alignment score measures how relevant each encoder state is to the decoder's current state, using a small learned network.
From theory to code
Implement additive_scores(s, H, W, U, v), which returns one score per encoder state: v . tanh(W s + U h_j).
Constraints
Hhas shape(T, d_h).
Hints
Hint 1
Broadcast the decoder state against every encoder row, apply tanh, then dot with v.
Keywords
research-paperstransformersllmattention
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.