Lookahead: The Slow Trajectory
Intermediate
The problem, from first principles
Across many synchronizations the slow weight behaves like an exponential moving average of the fast weights. Each sync pulls it toward the latest fast point, and the pull carries forward.
From theory to code
Implement slow_trajectory(phi0, fast_points, alpha), the sequence of slow weights over the syncs.
Constraints
- Carry the slow weight forward between syncs.
Hints
Hint 1
Loop over the fast points, applying the sync update to the running slow value, and record it.
Keywords
research-papersoptimizationmeta-optimizer
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.