Metraipolis › archive › dc_c5bc589ff6cb

Measuring the Energy Cost of Completing Tasks with Quantized Language-Model Agents — arXiv:2610.32417

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32417
License (as archived)
CC0-1.0
Archived
2026-10-26T19:55:39.101Z
SHA-256 of the archived text
70b5da40b365cd9955cc619263b7406e9d39b06e59168a9a427eea739b17f23f
Stable URL of this page
https://metraipolis.com/documento/dc_c5bc589ff6cb

How to cite

Measuring the Energy Cost of Completing Tasks with Quantized Language-Model Agents — arXiv:2610.32417 — arXiv (en), https://arxiv.org/abs/2610.32417 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_c5bc589ff6cb (Metraipolis archive dc_c5bc589ff6cb, sha256 70b5da40b365cd9955cc619263b7406e9d39b06e59168a9a427eea739b17f23f)
BibTeX
@misc{dcc5bc589ff6cb,
  title = {Measuring the Energy Cost of Completing Tasks with Quantized Language-Model Agents — arXiv:2610.32417},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32417},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_c5bc589ff6cb; sha256 70b5da40b365cd9955cc619263b7406e9d39b06e59168a9a427eea739b17f23f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32417. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Quantization reduces the memory and energy needed to serve large language models (LLMs) and AI agents for coding and data analysis can now run on locally hosted LLMs. In practice, this benefit is commonly benchmarked per output token. Yet an agent calls its model repeatedly, runs tools, and revises its work, and quantization can change both how much the agent generates and whether it succeeds. Existing energy studies either compare quantized models on single requests or measure agents at one fixed precision, overlooking how a change in precision plays out across a complete agent task. To address this gap, this paper presents the first characterization of the task-level energy trade-offs of model quantization in locally hosted agents. We develop 29 agent tasks with automatic pass/fail checks in six families, ranging from single-answer SQL analytics to multi-step repository bug fixing, and measure CPU and GPU energy per output token and per completed task for 4-bit and 8-bit versions of two open-weight models on bare-metal NVIDIA A100 nodes with this http URL . Our findings reveal that 4-bit precision generally saves energy per token, though whether it saves energy per completed task is task dependent and changes with the model and the execution budget (i.e., limits on turns, tokens, and time). We trace this discrepancy to the extra tokens needed to complete a task in full, which translates into using up to 3.5 times as much energy per completed task. These findings offer actionable guidance for developers and operators choosing model precision for agents, showing how tokens per completed task, set against the per-token saving, reveal whether lower precision completes the work with less energy.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.PF, nuovi lavori del giorno).
Categorie arXiv: Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.32417 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:46:33.362Z dalla pagina https://arxiv.org/list/cs.PF/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).