AdamW: Bias Correction
Beginner
The problem, from first principles
Adam's moment estimates start at zero, so they underestimate the true average early in training. AdamW keeps Adam's bias correction, which divides each moment by one minus its decay raised to the step count.
From theory to code
Implement bias_corrected(m, v, beta1, beta2, t), returning the corrected first and second moments.
Constraints
- The step count starts at 1.
Hints
Hint 1
Divide the first moment by one minus beta1 to the t, and the second by one minus beta2 to the t.
Keywords
research-papersoptimizationadam
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.