Gradient Clipping: The Clip Scale
Beginner
The problem, from first principles
The clip multiplier is the scalar that clip-by-norm applies. Separating it out makes the rule explicit: it is one when the norm is within bounds and shrinks the gradient otherwise.
From theory to code
Implement clip_scale(norm, threshold), the scalar clipping multiplier.
Constraints
- The multiplier never exceeds one.
Hints
Hint 1
Take the minimum of one and the threshold divided by the norm.
Keywords
research-papersoptimizationstability
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.