SimCLR: The NT-Xent Loss for One Anchor
Advanced
The problem, from first principles
SimCLR's contrastive loss, NT-Xent, asks for each anchor to pick its positive out of all the other views in the batch. It is a softmax cross-entropy over cosine similarities scaled by a temperature.
From theory to code
Implement nt_xent_anchor(sim_pos, sims_neg, tau), the softmax loss of the positive among the candidates.
Constraints
- Use
logaddexpfor numerical stability.
Hints
Hint 1
Stack the positive and negative similarities divided by tau, then compute logsumexp minus the positive's logit.
Keywords
research-papersunsupervisedcontrastiveself-supervised
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.