Adaptive Computation Time: The Weighted Output
Advanced
The problem, from first principles
The ACT output is a weighted average of the states after each step, where the weights are the halting probabilities and the last step takes the remainder. The output is differentiable with respect to the halting decision.
From theory to code
Implement act_output(outs, halts), the halting-weighted output.
Constraints
- Weights sum to one.
Hints
Hint 1
Use the halting probabilities for all steps but the last, give the last step the remainder, and take the weighted sum.
Keywords
research-paperssequence-modelsadaptive-computationrecurrence
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.