Metraipolis › archive › dc_22fff424fd73

Align, Then Correct: Training-Free Two-Stage Low-Rank Compensation for Extremely Quantized Large Language Models — arXiv:2610.08164

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.08164
License (as archived): CC0-1.0
Archived: 2026-10-07T11:23:48.805Z
SHA-256 of the archived text: 9af3f620112d0ef4d67c4fbda8741d223da2a41f52b7cafcb6a432d4ac4a1294
Stable URL of this page: https://metraipolis.com/documento/dc_22fff424fd73

How to cite

Align, Then Correct: Training-Free Two-Stage Low-Rank Compensation for Extremely Quantized Large Language Models — arXiv:2610.08164 — arXiv (en), https://arxiv.org/abs/2610.08164 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_22fff424fd73 (Metraipolis archive dc_22fff424fd73, sha256 9af3f620112d0ef4d67c4fbda8741d223da2a41f52b7cafcb6a432d4ac4a1294)

BibTeX

@misc{dc22fff424fd73,
  title = {Align, Then Correct: Training-Free Two-Stage Low-Rank Compensation for Extremely Quantized Large Language Models — arXiv:2610.08164},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08164},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_22fff424fd73; sha256 9af3f620112d0ef4d67c4fbda8741d223da2a41f52b7cafcb6a432d4ac4a1294},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08164. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Low-rank quantization error compensation (LQEC) recovers the accuracy lost under aggressive weight quantization by attaching a closed-form rank-$r$ adapter beside each frozen quantized weight, without any training. We show that existing compensators are limited by two shared simplifications. They calibrate symmetrically, evaluating the full-precision and compensated weights on the same activation, which yields a compensation target that is inherently high-rank -- so a fixed rank budget captures only a small fraction of it. And they minimize only the second-order term of the loss, although the compensated model is not stationary: a first-order descent direction larger than the applied compensation itself remains in every layer, and no reconstruction objective can absorb it. We propose a two-stage closed-form framework that removes both simplifications. Stage 1 aligns each layer's output with the full-precision model under a Fisher-weighted asymmetric objective, concentrating the rank budget on a rank-compressible target. Stage 2 re-measures statistics on the compensated model and applies a rank-constrained natural-gradient step that absorbs the remaining first-order signal. Every adapter is the result of a single truncated SVD; backward passes serve only to collect statistics. At 2 bits under QuIP#, our method reduces WikiText-2 perplexity from 12.43 to 10.26 on Qwen3-8B and from 21.11 to 13.22 on Qwen3-4B. On the held-out C4 corpus, it recovers 51% and 84% of the gap to FP16, versus 31% and 63% for the strongest baseline, with consistent gains in the seven-task zero-shot average, at higher bit-widths, and under a distinct quantizer.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.08164 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:15:39.499Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).