Metraipolis › archive › dc_d4dfab36503c
When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization — arXiv:2610.11226
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.11226
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T05:47:32.624Z
- SHA-256 of the archived text
c23f78de17b14528404ed27e44a5b94dd29658646c83541b805b57f96251323d- Stable URL of this page
https://metraipolis.com/documento/dc_d4dfab36503c
How to cite
When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization — arXiv:2610.11226 — arXiv (en), https://arxiv.org/abs/2610.11226 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d4dfab36503c (Metraipolis archive dc_d4dfab36503c, sha256 c23f78de17b14528404ed27e44a5b94dd29658646c83541b805b57f96251323d)
BibTeX
@misc{dcd4dfab36503c,
title = {When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization — arXiv:2610.11226},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.11226},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d4dfab36503c; sha256 c23f78de17b14528404ed27e44a5b94dd29658646c83541b805b57f96251323d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11226. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Weight-only post-training quantization (PTQ) relies heavily on reconstruction loss minimization to preserve model quality at low precision. We show that the weights favored by minimizing this loss need not yield better model performance on new tasks. In fact, we find that lower reconstruction loss can even degrade model performance on the same calibration data. Our analysis further shows that weights with lower reconstruction loss on calibration data can have higher loss than other weights when the distribution of input activations changes. Motivated by these observations and our analysis, we propose Distributionally Robust Quantization (DRQ), a post-hoc refinement process that minimizes worst-case reconstruction loss over a constrained set of input activation distributions. DRQ refines the integer codes representing quantized weights within the existing quantization grid, keeping quantization parameters and inference operators unchanged. Extensive experiments show that DRQ improves models quantized by six representative PTQ methods, including AWQ, GPTQ, and ParoQuant, and delivers gains across both dense and mixture-of-experts large language models. These results establish DRQ as a general post-hoc refinement framework for weight-only PTQ, achieving better downstream performance without adding inference overhead. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Machine Learning (stat.ML). Pagina del lavoro: https://arxiv.org/abs/2610.11226 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:47:00.596Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).