Per-Channel Weight Quantization for Linear Layers

The problem, from first principles

[01-symmetric-int8-quantization]'s single global scale for an entire weight matrix is dominated by whichever row happens to have the largest values — every OTHER row is then quantized far more coarsely than it needs to be, wasting precision. Implement per-channel (per-output-row) symmetric INT8 quantization for a weight matrix: instead of one global scale for the whole matrix, compute an independent scale for each output channel (row).

From theory to code

scale[o] = max(|W[o, :]|) / 127     for each output channel o
Q[o, :] = clip(round(W[o, :] / scale[o]), -127, 127)

Constraints

One independent scale per output row (axis=0 of the weight matrix), computed exactly as in symmetric per-tensor quantization but restricted to that row.
Rows with all-zero weights get scale 0 and an all-zero quantized row.
Return the quantized integer matrix, the per-row scale vector, and the dequantized reconstruction.

Hints

Hint: Vectorize over rows, don't loop

Compute np.max(np.abs(W), axis=1) to get one max-abs value per row in a single vectorized call. Broadcast each row's scale over its own row when quantizing: W / scale[:, None], not W / scale.