BYOL: The Normalized Regression Loss
Intermediate
The problem, from first principles
BYOL (Grill et al., 2020) learns without negative pairs at all. An online network predicts the output of a slowly updated target network for another view of the same image. The loss is a normalized regression, which equals two minus twice the cosine similarity.
From theory to code
Implement byol_loss(p, z), returning 2 - 2 cos(p, z).
Constraints
- Divide by both norms; no zero vectors.
Hints
Hint 1
Compute the cosine similarity of the two vectors and apply 2 - 2 cos.
Keywords
research-papersunsupervisedself-supervisedbootstrap
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.