Layer Normalization: Normalizing Across Features
Beginner
The problem, from first principles
Batch norm needs a batch of examples to compute statistics, which is awkward for sequence models and batch size 1. Layer normalization (Ba, Kiros & Hinton, 2016) computes the statistics across the features of each example instead.
From theory to code
Implement normalize_last_axis(x, eps), which normalizes each example over its feature dimension.
Constraints
- Works on any shape
(..., D). - The statistics are per example, not per batch.
Hints
Hint 1
Use axis=-1 with keepdims=True so each example's mean and variance broadcast back correctly.
Keywords
research-papersnormalizationlayernorm
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.