Metraipolis › archive › dc_ae0b423fc33e
Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs — arXiv:2608.18578
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.18578
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T13:53:48.433Z
- SHA-256 of the archived text
6e8a183899d2a7a38f531efbc88adec23cb94e5b598daeed2f81fa67628ce48f- Stable URL of this page
https://metraipolis.com/documento/dc_ae0b423fc33e
How to cite
Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs — arXiv:2608.18578 — arXiv (en), https://arxiv.org/abs/2608.18578 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ae0b423fc33e (Metraipolis archive dc_ae0b423fc33e, sha256 6e8a183899d2a7a38f531efbc88adec23cb94e5b598daeed2f81fa67628ce48f)
BibTeX
@misc{dcae0b423fc33e,
title = {Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs — arXiv:2608.18578},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.18578},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ae0b423fc33e; sha256 6e8a183899d2a7a38f531efbc88adec23cb94e5b598daeed2f81fa67628ce48f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.18578. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Proactive interference (PI) is a documented failure mode in large language models in which retrieval of a repeatedly overwritten value degrades as prior overwrites accumulate, mirroring a classical phenomenon in human working memory. Post-training quantization (PTQ) is now the default deployment path for open-weight models, yet its effect on this failure mode has not been tested. We evaluate three precision levels (FP16, INT8, INT4/NF4, via bitsandbytes) across three architecturally distinct instruction-tuned models (Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Phi-3.5-mini-instruct), holding the retrieval task fixed. INT4 quantization significantly reduces accuracy under high interference in every model (e.g., from 81.0% to 68.3% for Qwen), confirmed by paired McNemar's tests ($p \le 2.6 \times 10^{-6}$) and a mixed-effects regression spanning all interference levels; INT8, often assumed safe, also carries a smaller but real penalty in two of three models. The effect is specific to semantically similar (word-type) distractors and reverses sign under a numeric control condition, and is mechanistically linked to a rise in same-key intrusion errors under INT4 (from 21.5% to 24.6% of trials, $p = 4.8 \times 10^{-7}$). A follow-up ablation shows the effect originates in the quantized transformer backbone rather than the output projection layer. These results suggest that bitsandbytes 4-bit quantization can impose an additional cost on applications relying on long, updatable, semantically dense contexts, even when aggregate benchmark accuracy appears largely unaffected. We release our code and tokenizer-verified vocabulary construction method at this https URL
Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-20 in cs.LG).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.18578 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T13:50:58.996Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-20?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).