Metraipolis › archive › dc_c2b44fe4b5bc

Quantization Degradation in Large Language Models: A Signal-Noise Perspective — arXiv:2608.08188

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08188
License (as archived)
CC0-1.0
Archived
2026-10-19T08:50:44.109Z
SHA-256 of the archived text
a0328dec712b7ee518a6c868691085511fecf33c2dfc303eb4efba67f16f995b
Stable URL of this page
https://metraipolis.com/documento/dc_c2b44fe4b5bc

How to cite

Quantization Degradation in Large Language Models: A Signal-Noise Perspective — arXiv:2608.08188 — arXiv (en), https://arxiv.org/abs/2608.08188 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c2b44fe4b5bc (Metraipolis archive dc_c2b44fe4b5bc, sha256 a0328dec712b7ee518a6c868691085511fecf33c2dfc303eb4efba67f16f995b)
BibTeX
@misc{dcc2b44fe4b5bc,
  title = {Quantization Degradation in Large Language Models: A Signal-Noise Perspective — arXiv:2608.08188},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08188},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c2b44fe4b5bc; sha256 a0328dec712b7ee518a6c868691085511fecf33c2dfc303eb4efba67f16f995b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08188. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Post-training quantization reduces the deployment cost of large language models, yet how severely a quantized model degrades is not determined by bit-width alone. We systematically study weight-only post-training quantization across bit-widths, quantization methods, model scales and downstream tasks on multiple model families. We observe that such degradation varies substantially across these factors: 4-bit quantization usually preserves performance, 2-bit often causes broad degradation, and at 3-bit, degradation becomes apparent but varies markedly with task type, quantization method and model scale. To explain this variability, we use the signal-to-noise ratio (SNR) to measure how strongly quantization perturbs full-precision representations. We trace degradation back to two linked processes: how quantization errors arise within individual modules, and how they accumulate across layers. First, a source SNR decomposition shows that newly introduced errors depend on three factors: the magnitude of the weight error, the strength of the task-specific signal, and how strongly the quantization error aligns with task-specific activations. Different factors affect these components in distinct ways. Second, a cross-layer propagation analysis shows that these errors can be attenuated, preserved, or amplified as they pass across layers, and that larger models benefit from weaker error amplification. Together, these results establish that quantization degradation is governed by how errors are introduced at the source and how they accumulate across the network.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.08188 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.974Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).