Quantize a Full Weight Matrix, Measure Size/Accuracy Tradeoff
The problem, from first principles
01/02 built quantize and dequantize as isolated operations. The actual reason anyone quantizes a real model is a tradeoff: smaller storage and faster integer arithmetic, in exchange for some amount of accuracy loss — and "some amount" needs to be an actual measured number, not just an assumption, before shipping a quantized model.
From theory to code
Implement quantize_weight_matrix(weight), which quantizes a whole weight matrix with one shared scale/zero_point, then reports both sides of the tradeoff: the compression ratio achieved, and the reconstruction error that compression cost.
Constraints
weight: any-shape float32 array (e.g. a real layer's weight matrix).- Returns a dict with
quantized,scale,zero_point,compression_ratio,max_abs_error,mean_abs_error. compression_ratioassumes the original was stored asfloat32(4 bytes/element) and the quantized version asint8(1 byte/element) — always exactly4.0for this dtype pair, regardless of the matrix's actual values.max_abs_error/mean_abs_error: plain Python floats, computed against the dequantized reconstruction.
Hints
Open one at a time. Each gives away a little more than the last.
Hint 1
This is almost entirely composition: call 01's quantize and 02's dequantize, and this question's own job is just computing and packaging the tradeoff numbers around them.
Hint 2
compression_ratio doesn't depend on the matrix's actual content at all — it's a fixed 4.0 from 4 bytes / 1 byte, the same for every float32-to-int8 quantization regardless of shape or values.
Keywords
No results yet
Run your code to check it against the examples, or submit to run every test.