MoCo: The Momentum Encoder Update
Beginner
The problem, from first principles
MoCo (He et al., 2020) keeps a queue of negative keys produced by a slowly moving encoder. Using a momentum update rather than gradients keeps the keys consistent across the queue, which is what makes the queue usable.
From theory to code
Implement momentum_update(key, query, m), returning the momentum-weighted blend of the key and query parameters.
Constraints
mis close to 1, typically 0.999.
Hints
Hint 1
Weight the key parameters by m and the query parameters by 1 - m.
Keywords
research-papersunsupervisedcontrastivemomentum
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.