Deep InfoMax: The Jensen-Shannon MI Estimate
Advanced
The problem, from first principles
Deep InfoMax (Hjelm et al., 2019) maximizes mutual information between an input's local features and its global summary, by training a discriminator to tell matched pairs from mismatched ones. The Jensen-Shannon form is a stable estimate built from softplus terms.
From theory to code
Implement jsd_estimate(pos, neg), the difference of the mean softplus terms over positive and negative pairs.
Constraints
- Use
logaddexp(0, x)for softplus.
Hints
Hint 1
Positive pairs contribute -softplus(-s), negative pairs contribute -softplus(s).
Keywords
research-papersunsupervisedmutual-informationrepresentation
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.