Cyclical Learning Rates: The LR Range Test
Beginner
The problem, from first principles
The LR range test (Smith, 2015) raises the learning rate exponentially over a short run and records the loss. The rate where the loss falls fastest is a good choice for the bounds of the cyclical schedule.
From theory to code
Implement lr_range_test(it, min_lr, gamma), the geometrically increasing learning rate.
Constraints
- Iterations start at zero.
Hints
Hint 1
Multiply the starting rate by the growth factor raised to the iteration number.
Keywords
research-papersoptimizationlearning-rate-schedule
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.