Maxout Networks: The Maxout Unit
Intermediate
The problem, from first principles
Maxout networks (Goodfellow et al., 2013) make the activation itself learnable: each unit outputs the maximum of several linear functions of its input. The paper shows this family can approximate any convex function and works well with dropout.
From theory to code
Implement maxout_forward(x, W, b, k), computing out * k linear pieces and returning the maximum over each group of k.
Constraints
Whasout * krows.- Pieces for the same unit are adjacent.
Hints
Hint 1
Compute all pieces with one affine map, reshape to (N, out, k), then take the max over the last axis.
Keywords
research-papersactivationmaxout
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.