Weight Normalization: A Linear Layer
Intermediate
The problem, from first principles
A weight-normalized layer is an ordinary linear layer whose weight matrix is built from g and v. Using it as a drop-in replacement shows the reparameterization is just a change of variables.
From theory to code
Implement weight_norm_linear(x, v, g, b), which builds W with weight normalization and returns x W^T + b.
Constraints
- Output shape is
(N, out).
Hints
Hint 1
Build W the same way as weight_norm, then apply the affine map.
Keywords
research-papersnormalizationweight-norm
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.