Chinchilla: The Compute-Optimal Token Count
Beginner
The problem, from first principles
Chinchilla (Hoffmann et al., 2022) found that many large models were trained on too few tokens for their size. For compute-optimal training, parameters and tokens should grow together, at roughly twenty tokens per parameter.
From theory to code
Implement chinchilla_optimal_tokens(n_params), returning 20 * N.
Constraints
- Result is a float.
Hints
Hint 1
Multiply the parameter count by 20.
Keywords
research-paperstransformersllmscalingcompute
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.