RNN Regularization: Truncated BPTT Chunks
Beginner
The problem, from first principles
Training an RNN on a long text is split into truncated segments, and gradients only flow back through a fixed window. The number of segments per pass decides how often the hidden state is carried over.
From theory to code
Implement bptt_chunks(T, bptt), the number of truncated segments needed for a sequence.
Constraints
- Round up for a partial last segment.
Hints
Hint 1
Use ceiling division of the length by the truncation window.
Keywords
research-paperssequence-modelslanguage-modellingregularization
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.