Scaling Laws: Predicting the Gain From Growth
Beginner
The problem, from first principles
Scaling laws are most useful for planning. If you know the exponent, you can estimate how much loss you will save by growing the model, before you spend the compute to train it.
From theory to code
Implement loss_ratio_for_size_increase(factor, alpha), returning factor ** (-alpha).
Constraints
factoris the ratio of new size to old size.
Hints
Hint 1
Raise the factor to the power of the negated exponent.
Keywords
research-paperstransformersllmscaling
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.