Cyclical Learning Rates: The Triangular Schedule
Intermediate
The problem, from first principles
Smith (2015) cycles the learning rate between a lower and upper bound instead of decaying it monotonically. A triangular wave goes up and down each cycle, which lets the model escape poor regions.
From theory to code
Implement triangular_lr(it, base, max_lr, step), the triangular cyclical rate.
Constraints
- Each half-cycle takes
stepiterations.
Hints
Hint 1
Find which cycle the iteration is in, measure its distance from the peak, and interpolate linearly between the bounds.
Keywords
research-papersoptimizationlearning-rate-schedule
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.