Adam: One Full Update With Bias Correction
The problem, from first principles
01-adam-moment-updates built the two running averages. A full Adam step still needs two more things. First, the averages start at zero, so early steps underestimate them; Adam corrects for this. Second, the step itself is the bias-corrected first moment divided by the square root of the bias-corrected second moment. Together these produce a step size that adapts to each parameter.
From theory to code
Implement adam_step(theta, grad, m, v, t, lr, beta1, beta2, eps). It updates the moments, bias-corrects them, applies the update, and returns (theta, m, v). The signature and docstring are already in the editor.
Constraints
tis 1-based: the first call usest=1.- Defaults are the paper's:
lr=0.001,beta1=0.9,beta2=0.999,eps=1e-8. - Works on scalars and NumPy arrays.
Hints
Hint 1
Reuse the moment update from the previous question, then divide each moment by 1 - beta**t before using it.
Hint 2
The parameter moves by lr * m_hat / (sqrt(v_hat) + eps). Subtract that from theta; the minus sign points the step against the gradient.
Keywords
No results yet
Run your code to check it against the examples, or submit to run every test.