GPTQ: Symmetric Quantization
Intermediate
The problem, from first principles
GPTQ (Frantar et al., 2022) quantizes weights to a few bits after training. The simplest building block is symmetric round-to-nearest quantization: pick a scale from the largest weight, then round each weight to a small integer grid.
From theory to code
Implement quantize_symmetric(x, bits), returning integer codes and the scale.
Constraints
- Use a scale of one when all weights are zero.
Hints
Hint 1
Set the scale to the largest magnitude divided by the largest code, round the scaled weights, and clip to the code range.
Keywords
research-paperssystemsquantizationinference
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.