Squeeze-and-Excitation: The Excitation Gate
Intermediate
The problem, from first principles
The excitation step turns the channel summary into a gate for each channel. A small bottleneck with ReLU and a sigmoid output learns which channels to amplify and which to suppress.
From theory to code
Implement excitation(s, W1, W2), returning the sigmoid of the second projection applied to the ReLU of the first.
Constraints
- The bottleneck width is C divided by a reduction ratio.
Hints
Hint 1
Project down with W1, apply ReLU, project back up with W2, then take the sigmoid.
Keywords
research-paperscomputer-visionattentionarchitecture
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.