Gradient Clipping: Counting Exploding Steps
Beginner
The problem, from first principles
Clipping only helps if you know how often it fires. Counting the steps whose gradient norm exceeds the threshold tells you whether the threshold is set too low or the run is unstable.
From theory to code
Implement exploding_steps(grad_norms, threshold), the count of steps above the threshold.
Constraints
- Strictly greater than the threshold counts.
Hints
Hint 1
Count the norms that are larger than the threshold.
Keywords
research-papersoptimizationstability
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.