Bahdanau Attention: The Context Vector
Beginner
The problem, from first principles
Once the weights are known, the context vector is the weighted average of the encoder states. It gives the decoder one summary of the input that is tailored to the word it is about to produce.
From theory to code
Implement context_vector(weights, H), which returns the weighted sum of the encoder states.
Constraints
- Weights sum to 1 across the
Tpositions.
Hints
Hint 1
Use a matrix product: weights of shape (T,) times states of shape (T, d).
Keywords
research-paperstransformersllmattention
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.