Dropout: Backpropagating Through the Mask
Beginner
The problem, from first principles
Dropout zeroes random units during training, so the backward pass must treat those units the same way: a unit that was dropped in the forward pass contributed nothing to the output, so it receives no gradient.
From theory to code
Implement dropout_backward(grad_out, keep_mask, p), which multiplies the incoming gradient by the same mask and rescale used in the forward pass.
Constraints
keep_maskhas the same shape asgrad_out.0 <= p < 1.
Hints
Hint 1
Use the exact mask and scale from the forward pass. Dropped units get a gradient of zero; kept units get the same 1 / (1 - p) factor.
Keywords
research-papersregularizationdropout
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.