The Lottery Ticket Hypothesis: Magnitude Pruning Mask
Intermediate
The problem, from first principles
Frankle & Carbin (2019) found that a sparse subnetwork, chosen by pruning the smallest-magnitude weights after training, can train to the same accuracy as the full network from its original initialization. The first step is choosing which weights to keep.
From theory to code
Implement magnitude_mask(w, keep_fraction), which keeps the largest-magnitude weights and returns a boolean mask.
Constraints
- Keep
ceil(keep_fraction * size)weights.
Hints
Hint 1
Sort the absolute values in descending order, mark the top k as True, and reshape to the weights' shape.
Keywords
research-papersclassical-mlpruninglottery-ticket
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.