Metraipolis › archive › dc_b422c7d25a4e
Activation Denoising: A Robustness View on Parallel vs Sequential LLM Quantization — arXiv:2610.07522
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.07522
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T11:41:14.073Z
- SHA-256 of the archived text
6c296ccf1406ebf5a870bd0b584d727b8cecd62d05097145a1d73c01a4c42f66- Stable URL of this page
https://metraipolis.com/documento/dc_b422c7d25a4e
How to cite
Activation Denoising: A Robustness View on Parallel vs Sequential LLM Quantization — arXiv:2610.07522 — arXiv (en), https://arxiv.org/abs/2610.07522 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b422c7d25a4e (Metraipolis archive dc_b422c7d25a4e, sha256 6c296ccf1406ebf5a870bd0b584d727b8cecd62d05097145a1d73c01a4c42f66)
BibTeX
@misc{dcb422c7d25a4e,
title = {Activation Denoising: A Robustness View on Parallel vs Sequential LLM Quantization — arXiv:2610.07522},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.07522},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b422c7d25a4e; sha256 6c296ccf1406ebf5a870bd0b584d727b8cecd62d05097145a1d73c01a4c42f66},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.07522. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Post-training quantization is a powerful tool for compressing large language models. The most scalable methods quantize every layer in parallel, but quantization errors then compound through the residual stream, as no layer corrects for the errors of the layers before it. Sequential quantization accounts for this error compounding by re-calibrating each layer on the already-quantized outputs of its predecessors, yielding stronger results but at the cost of a serial schedule that becomes a bottleneck at scale. As a solution, we propose parallel quantization with activation denoising, which recovers much of the sequential benefit while keeping quantization fully parallel. Rather than re-calibrating layer-by-layer, we take a robustness perspective and model the upstream error as noise, regularizing to be robust to it through a preprocessing step followed by metric-weighted rounding. Applied at every layer, this regularization forms a depth-compounding smoothness penalty that dampens how strongly quantization errors amplify through the model. Unlike orthogonal rotations commonly used in quantization, which must preserve the model's function, we multiply the weights by a more general linear transformation. We find that the two are complementary and their effects compound. Empirically, our robustness regularization recovers a significant part of sequential quantization's benefit in a single parallel pass, at a fraction of its time. Overall, by treating compounding quantization errors as a robustness problem, we offer a principled foundation for more efficient and accurate LLM quantization at scale. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.07522 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T11:19:24.768Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).