CPC: The Contrastive Loss
Intermediate
The problem, from first principles
Contrastive Predictive Coding (van den Oord et al., 2018) predicts future latent codes from the present context. Among many candidate futures, the true one should get the highest score, which is a softmax classification problem.
From theory to code
Implement cpc_loss(scores, pos), the softmax cross-entropy of the true future among the candidates.
Constraints
- Use
logsumexpfor stability.
Hints
Hint 1
Compute logsumexp(scores) - scores[pos].
Keywords
research-papersunsupervisedpredictive-codingcontrastive
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.