Show, Attend and Tell: Sampling Hard Attention
Advanced
The problem, from first principles
The hard variant of attention in Show, Attend and Tell samples one location per word rather than averaging. It is not differentiable, so the paper trains it with a score-function gradient estimator.
From theory to code
Implement sample_hard_attention(alpha, rng), drawing one location from the attention distribution.
Constraints
- Use the supplied
rngfor reproducibility.
Hints
Hint 1
Call the generator's choice with the attention distribution as probabilities.
Keywords
research-paperssequence-modelsattentioncaptioning
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.