Scaling Laws: Fitting the Exponent
Intermediate
The problem, from first principles
Given measured losses at several model sizes, a power law can be fitted by a straight-line regression in log-log space. The slope of that line is minus the exponent, which tells you how fast returns to scale diminish.
From theory to code
Implement fit_power_law_exponent(N, L), returning the exponent alpha from a least-squares line through log N and log L.
Constraints
- Returns a Python float.
Hints
Hint 1
Use np.polyfit on the logs with degree 1, then return the negative of the slope.
Keywords
research-paperstransformersllmscaling
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.