Metraipolis › archive › dc_28dcebc82cf6
Beyond Accuracy: How Quantization Affects Coding Agents — arXiv:2611.02368
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02368
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T10:48:17.499Z
- SHA-256 of the archived text
283f75570bc86e66d352c0e806913b5c288a78e62b76ff9c4fe1e64757d189ee- Stable URL of this page
https://metraipolis.com/documento/dc_28dcebc82cf6
How to cite
Beyond Accuracy: How Quantization Affects Coding Agents — arXiv:2611.02368 — arXiv (en), https://arxiv.org/abs/2611.02368 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_28dcebc82cf6 (Metraipolis archive dc_28dcebc82cf6, sha256 283f75570bc86e66d352c0e806913b5c288a78e62b76ff9c4fe1e64757d189ee)
BibTeX
@misc{dc28dcebc82cf6,
title = {Beyond Accuracy: How Quantization Affects Coding Agents — arXiv:2611.02368},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02368},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_28dcebc82cf6; sha256 283f75570bc86e66d352c0e806913b5c288a78e62b76ff9c4fe1e64757d189ee},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02368. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Agents are increasingly used for coding tasks, but the inference cost of large language models (LLMs) limits where they can run. Quantization reduces model size, but its effects on coding agents that execute and revise programs remain underexplored. We compare BF16 with 4-bit AWQ and GPTQ for Qwen2.5-Coder-Instruct (7B, 14B, and 32B) and Devstral-Small-2507 (24B) on OJBench coding problems across three seeds. We detect no statistically significant accuracy loss for any of the coding agents. However, final accuracy can conceal substantial behavioral changes: AWQ reduces Qwen-7B one-shot solves from 17 to 0 while increasing loop-based solves from 10 to 25. All Qwen-32B successes involve tool use, whereas Devstral rarely makes a valid executor call. Failure profiles also differ: Qwen-32B produces far more compilation errors than the smaller Qwen models, rising slightly under quantization, while Devstral's failures shift significantly from runtime errors toward wrong answers and timeouts. These findings indicate that quantization robustness in coding agents should be assessed through solution trajectories, tool use, and failure modes, in addition to final accuracy. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.02368 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:39:50.562Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).