CPC: The Mutual Information Bound
Beginner
The problem, from first principles
The CPC paper shows that the loss bounds the mutual information between context and future from below. The bound equals log N minus the loss, so a better classifier gives a tighter bound on the information it captures.
From theory to code
Implement mi_lower_bound(n_candidates, loss), returning log(N) - loss.
Constraints
- Returns a Python float.
Hints
Hint 1
Compute the natural log of the candidate count and subtract the loss.
Keywords
research-papersunsupervisedpredictive-codingcontrastive
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.