Adaptive Computation Time: The Ponder Cost
Beginner
The problem, from first principles
Without a penalty, ACT would think for as long as possible. The ponder cost adds a small price per step, so the network thinks only as long as the extra computation helps.
From theory to code
Implement act_ponder_cost(n_steps, tau), the penalty added to the training loss.
Constraints
- Returns a float.
Hints
Hint 1
Multiply the step count by the penalty weight.
Keywords
research-paperssequence-modelsadaptive-computationrecurrence
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.