BYOL: The Target Network Update
Beginner
The problem, from first principles
BYOL's target network is not trained by gradients. After every step it moves a small fraction toward the online network, which gives the target a slowly changing, stable identity.
From theory to code
Implement ema_update(target, online, tau), returning (1 - tau) * target + tau * online.
Constraints
tauis small in practice, such as 0.004 or 0.01.
Hints
Hint 1
Blend the two parameter arrays with weights 1 - tau and tau.
Keywords
research-papersunsupervisedself-supervisedbootstrap
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.