Shampoo: The Inverse Root
Intermediate
The problem, from first principles
Shampoo scales the gradient by the inverse root of each preconditioner's eigenvalues. Raising eigenvalues to minus one over the root order gives the matrix power that Shampoo needs.
From theory to code
Implement inverse_root(eigs, power), the element-wise inverse root of positive eigenvalues.
Constraints
- Work element-wise on the eigenvalues.
Hints
Hint 1
Raise the eigenvalues to the power minus one over the root order.
Keywords
research-papersoptimizationpreconditioning
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.