Dropout: Averaging Over Masks Recovers the Input
Intermediate
The problem, from first principles
Dropout is random, so a single forward pass is noisy. The paper's key claim is that the average over many dropout masks behaves like the un-dropped network with the right scaling. This question checks that claim numerically.
From theory to code
Implement dropout_expectation(x, p, n_samples, rng) that draws n_samples masks, applies inverted dropout to each, and returns the mean.
Constraints
rngis anumpy.random.Generator.n_samplesis a positive integer.
Hints
Hint 1
Draw all masks at once with rng.random((n_samples,) + x.shape) >= p, then average over axis 0.
Keywords
research-papersregularizationdropout
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.