Metraipolis › archive › dc_a2c24e0454ea
Why Does Post-Training Quantization Work? — arXiv:2609.11716
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.11716
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T05:44:29.446Z
- SHA-256 of the archived text
4aeb9427f8b520c209e5149fcd665b57d32a4d3c95d3dbc9a86a6259fba25235- Stable URL of this page
https://metraipolis.com/documento/dc_a2c24e0454ea
How to cite
Why Does Post-Training Quantization Work? — arXiv:2609.11716 — arXiv (en), https://arxiv.org/abs/2609.11716 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a2c24e0454ea (Metraipolis archive dc_a2c24e0454ea, sha256 4aeb9427f8b520c209e5149fcd665b57d32a4d3c95d3dbc9a86a6259fba25235)
BibTeX
@misc{dca2c24e0454ea,
title = {Why Does Post-Training Quantization Work? — arXiv:2609.11716},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.11716},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a2c24e0454ea; sha256 4aeb9427f8b520c209e5149fcd665b57d32a4d3c95d3dbc9a86a6259fba25235},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11716. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Post-training quantization compresses large language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces an error into the hidden states. Naively, these errors should accumulate with depth and corrupt next-token prediction; randomly initialized models accumulate these discrepancies rapidly, whereas quantized pretrained models accumulate much less hidden-state error and largely maintain downstream task performance, even though they were never trained with quantization noise. This raises the question we address: why does post-training quantization work? Comparing full-precision and quantized forward passes, we identify two mechanisms that characterize pretrained quantization robustness. First, the error a layer newly introduces tends to oppose the error it inherits from the layer's input. The two cancel partially such that the discrepancy between full-precision and quantized passes grows slowly. This counteracting residual interaction develops during pretraining. Our quantitative analysis identifies it as a major factor slowing hidden-error growth. Second, LM-head geometry preferentially preserves the scores and probabilities of high-ranked tokens, which typically represent the model's most confident predictions. Together, these mechanisms explain why quantization error that passes through numerous layers can still produce only small output changes, and we verify the findings across models and quantization settings. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-11 in cs.CL). Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2609.11716 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T05:23:57.047Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).