Learning Phrase Representations: The Candidate State
Intermediate
The problem, from first principles
The candidate is what the unit would move to if it were fully updated. The reset gate lets the unit ignore the old state when computing that candidate, so it can start a fresh representation after an irrelevant part of the input.
From theory to code
Implement gru_candidate(x, h, r, W, U), which returns tanh(W x + U (r * h)).
Constraints
- The reset gate multiplies
helement-wise before the recurrent product.
Hints
Hint 1
Compute r * h first, apply U to it, add W x, then tanh.
Keywords
research-paperstransformersllmrecurrentgru
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.