Metraipolis › archive › dc_f3e9e2cc6952
Beyond Accuracy: How Quantization Affects Coding Agents — arXiv:2611.02368
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02368
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:37:34.581Z
- SHA-256 of the archived text
ea0b44b8ccc65e25a1773cc431c230c97f50ad163f87576637c437ade122ff3e- Stable URL of this page
https://metraipolis.com/documento/dc_f3e9e2cc6952
How to cite
Beyond Accuracy: How Quantization Affects Coding Agents — arXiv:2611.02368 — arXiv (en), https://arxiv.org/abs/2611.02368 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f3e9e2cc6952 (Metraipolis archive dc_f3e9e2cc6952, sha256 ea0b44b8ccc65e25a1773cc431c230c97f50ad163f87576637c437ade122ff3e)
BibTeX
@misc{dcf3e9e2cc6952,
title = {Beyond Accuracy: How Quantization Affects Coding Agents — arXiv:2611.02368},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02368},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f3e9e2cc6952; sha256 ea0b44b8ccc65e25a1773cc431c230c97f50ad163f87576637c437ade122ff3e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02368. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Agents are increasingly used for coding tasks, but the inference cost of large language models (LLMs) limits where they can run. Quantization reduces model size, but its effects on coding agents that execute and revise programs remain underexplored. We compare BF16 with 4-bit AWQ and GPTQ for Qwen2.5-Coder-Instruct (7B, 14B, and 32B) and Devstral-Small-2507 (24B) on OJBench coding problems across three seeds. We detect no statistically significant accuracy loss for any of the coding agents. However, final accuracy can conceal substantial behavioral changes: AWQ reduces Qwen-7B one-shot solves from 17 to 0 while increasing loop-based solves from 10 to 25. All Qwen-32B successes involve tool use, whereas Devstral rarely makes a valid executor call. Failure profiles also differ: Qwen-32B produces far more compilation errors than the smaller Qwen models, rising slightly under quantization, while Devstral's failures shift significantly from runtime errors toward wrong answers and timeouts. These findings indicate that quantization robustness in coding agents should be assessed through solution trajectories, tool use, and failure modes, in addition to final accuracy. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.02368 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:29:48.860Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).