Symmetric INT8 Quantization

The problem, from first principles

Storing and computing with 32-bit floats is expensive at inference time — most of that precision is unnecessary for a model that's already trained. Implement symmetric (zero-point-free) INT8 quantization and dequantization for a single tensor: find a scale that maps the tensor's largest-magnitude value to the INT8 range, quantize, and reconstruct an approximation.

From theory to code

scale = max(|x|) / 127
q = clip(round(x / scale), -127, 127)
x_hat = q * scale

Constraints

• scale = max(abs(x)) / 127, with scale = 0 handled explicitly when the tensor is all zeros.
• Quantized values are clipped to the inclusive range [-127, 127] as integers.
• Dequantized reconstruction is q * scale.
• Return the quantized integer tensor, the scale used, and the dequantized reconstruction.

Hints