Metraipolis › archive › dc_3de95d07cd41

Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals — arXiv:2609.21450

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.21450
License (as archived)
CC0-1.0
Archived
2026-10-10T01:45:06.350Z
SHA-256 of the archived text
658c81cc22798ff9cad143bc7758cb079758b2cd41df805c8bb86b27aa5050fc
Stable URL of this page
https://metraipolis.com/documento/dc_3de95d07cd41

How to cite

Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals — arXiv:2609.21450 — arXiv (en), https://arxiv.org/abs/2609.21450 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3de95d07cd41 (Metraipolis archive dc_3de95d07cd41, sha256 658c81cc22798ff9cad143bc7758cb079758b2cd41df805c8bb86b27aa5050fc)
BibTeX
@misc{dc3de95d07cd41,
  title = {Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals — arXiv:2609.21450},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.21450},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3de95d07cd41; sha256 658c81cc22798ff9cad143bc7758cb079758b2cd41df805c8bb86b27aa5050fc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.21450. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Post-training weight-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains difficult because activation outliers degrade effective quantization resolution. Although weight optimization, channel-wise scaling, and orthogonal rotation mitigate this problem, the error components they address and their relationship remain unclear. Using an exact decomposition of local weight-activation quantization error into an activation-guided weight compensation term and an orthogonal residual, we bound the residual using persistent channel-wise outlier and regular activation quantities. This decomposition clarifies which error components can be addressed by weight compensation and which require transformation design. We then use the residual bounds to derive practical guidelines for applying randomized Hadamard rotation, sign selection, and channel scaling. In particular, the analysis explains how random signs suppress constructive interference among persistent outlier channels, how sampling multiple sign patterns can improve transformation selection, and how second-moment balancing leads to an $L_2$ scaling rule while a further relaxation recovers SmoothQuant-style $L_\infty$ scaling. We evaluate these guidelines through backpropagation-free configurations across eight Llama and Mistral models, obtaining performance competitive with gradient-trained SpinQuant.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-21 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.21450 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T01:14:16.631Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-21?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).