Metraipolis › archive › dc_8fca5b89294f

Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs — arXiv:2608.08506

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.08506
License (as archived)
CC0-1.0
Archived
2026-10-19T08:50:49.349Z
SHA-256 of the archived text
4a096de50d4d5e48c8d5b8daa38a5aff2da0c130ac1881adcfc41647f5fb52ca
Stable URL of this page
https://metraipolis.com/documento/dc_8fca5b89294f

How to cite

Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs — arXiv:2608.08506 — arXiv (en), https://arxiv.org/abs/2608.08506 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8fca5b89294f (Metraipolis archive dc_8fca5b89294f, sha256 4a096de50d4d5e48c8d5b8daa38a5aff2da0c130ac1881adcfc41647f5fb52ca)
BibTeX
@misc{dc8fca5b89294f,
  title = {Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs — arXiv:2608.08506},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.08506},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8fca5b89294f; sha256 4a096de50d4d5e48c8d5b8daa38a5aff2da0c130ac1881adcfc41647f5fb52ca},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.08506. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Training-free low-rank compression frameworks have been gaining prominence for LLM compression given their effectiveness in reducing model parameter count while maintaining task-level accuracy. However, existing SOTA frameworks share two key limitations: (1) residual errors in calibration data activations accumulate across layers during compression, causing misalignment between representations simulated at compression time and those experienced at inference; (2) the assumption that layer importance distribution is preserved post-compression does not hold. Together, these two effects introduce misalignment in the compression process in relation to the deployed model. We study these effects and propose a simple, training-free methodology compatible with existing frameworks to mitigate them, comprising: (1) Layer-by-Layer Compression with Calibration Correction; (2) Iterative Compression with Rank Allocation Correction. Implemented atop an existing SOTA decomposition framework, and evaluated on Llama and Qwen3 models across various benchmarks and compression rates, our approach demonstrates up to ~1-2.5 accuracy point improvements over per-weight and joint decomposition baselines on zero-shot tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CL).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2608.08506 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:49:41.987Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).