Instance Normalization

The problem, from first principles

Instance Normalization normalizes each instance (sample) and channel independently, ignoring batch statistics. Unlike BatchNorm which pools across the batch, InstanceNorm computes mean/variance per instance-channel pair.

\text{instance\_norm}(x, \gamma, \beta, \epsilon) = \gamma \frac{x - \mu_{nc}}{\sqrt{\sigma_{nc}^2 + \epsilon}} + \beta

Where mu_nc and sigma_nc are computed per instance and channel (across spatial dimensions).

From theory to code

Implement:

instance_norm(x, gamma, beta, eps=1e-5)

Constraints

x shape: (N, C, H, W) or (N, C)
gamma, beta shape (C,)
Return normalized x, same shape.

Instance Normalization

The problem, from first principles

From theory to code

Constraints

Keywords