Metraipolis › archive › dc_3dbba41c3ed8

PRQuant: Permutation Residual Quantization for Low-Overhead Inference — arXiv:2609.22106

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.22106
License (as archived)
CC0-1.0
Archived
2026-10-09T07:05:56.244Z
SHA-256 of the archived text
563b45eebfb27b1033f7ff0f8579aff4abaad8866402c2b7c1f04cddf921db5b
Stable URL of this page
https://metraipolis.com/documento/dc_3dbba41c3ed8

How to cite

PRQuant: Permutation Residual Quantization for Low-Overhead Inference — arXiv:2609.22106 — arXiv (en), https://arxiv.org/abs/2609.22106 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3dbba41c3ed8 (Metraipolis archive dc_3dbba41c3ed8, sha256 563b45eebfb27b1033f7ff0f8579aff4abaad8866402c2b7c1f04cddf921db5b)
BibTeX
@misc{dc3dbba41c3ed8,
  title = {PRQuant: Permutation Residual Quantization for Low-Overhead Inference — arXiv:2609.22106},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.22106},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3dbba41c3ed8; sha256 563b45eebfb27b1033f7ff0f8579aff4abaad8866402c2b7c1f04cddf921db5b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.22106. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Low-bit quantization of linear layers is often dominated by a small number of outlier channels. Existing smoothing, rotation, and residual-based methods can mitigate this issue, but may shift the quantization bottleneck to weights or introduce costly online operations. To address these limitations, we propose PRQuant (Permutation Residual Quantization), a training-free framework that combines channel permutation with offline weight residual compensation. PRQuant identifies the scaled-weight columns with the largest quantization errors and permutes them into contiguous tail blocks. This structure allows the corresponding weight residuals to be precomputed entirely offline, while replacing scattered activation gathering with simple contiguous access during inference, yielding a single regular MXFP4 GEMM for compensated computation. Experiments show that PRQuant substantially reduces down-projection reconstruction error, with scaling and residual compensation providing the main numerical gains while permutation enables a hardware-friendly contiguous layout. Comprehensive experiment results on Qwen3-4B-Instruct-2507 and Qwen3-30B-A3B-Instruct-2507 illustrate that PRQuant achieves up to averagelly 2.6x and 1.8x operator speedup over BF16 respectively, while preserving near plain MXFP4 end-to-end decoding efficiency. Across five downstream benchmarks, PRQuant achieves the best average accuracy among the quantized methods, improving accuracy over MXFP4 by 1.24 and 0.55, respectively.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.22106 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:58:09.109Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).