Metraipolis › archive › dc_7faf71fc3e1c
Understanding the Energy Cost of Quantized Agents on Shared GPUs — arXiv:2610.41672
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.41672
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T07:57:24.796Z
- SHA-256 of the archived text
409ae69f82b6115e9a503adf5af88084fd3c0467f356fd512051ac4772cc4aea- Stable URL of this page
https://metraipolis.com/documento/dc_7faf71fc3e1c
How to cite
Understanding the Energy Cost of Quantized Agents on Shared GPUs — arXiv:2610.41672 — arXiv (en), https://arxiv.org/abs/2610.41672 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7faf71fc3e1c (Metraipolis archive dc_7faf71fc3e1c, sha256 409ae69f82b6115e9a503adf5af88084fd3c0467f356fd512051ac4772cc4aea)
BibTeX
@misc{dc7faf71fc3e1c,
title = {Understanding the Energy Cost of Quantized Agents on Shared GPUs — arXiv:2610.41672},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.41672},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7faf71fc3e1c; sha256 409ae69f82b6115e9a503adf5af88084fd3c0467f356fd512051ac4772cc4aea},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41672. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Quantization can lower the energy of language-model calls and free memory for serving more agents, but its benefit per completed task remains unclear when agents share a GPU. Measurements of fixed requests or isolated agents leave open how the agents' generated work interact with competition for compute and cache. We present a systematic study of precision and admission limits for two agent models on A100 deployments with 40 or 80 GB of memory, using coding and data-analysis tasks with independently verified outcomes. We replay recorded calls across precisions to isolate execution energy and cap the cache to test the value of freed memory. For both models with four admitted agents where requests fit in the cache, 4-bit weights with 16-bit activations execute the same calls with 10-25% less energy than 8-bit weights and activations. Despite this saving, extra work and fewer completions in live runs raise their energy per verified completion by 49%. Where requests compete for cache on the 40 GB deployment, 4-bit weights cost less, and capping their cache raises energy per verified completion by 34% with 16 agents. These results show how generated work and cache demand change the energy benefit of quantization, providing a practical basis for choosing precision and admission limits together. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.PF, nuovi lavori del giorno). Categorie arXiv: Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2610.41672 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T07:50:30.905Z dalla pagina https://arxiv.org/list/cs.PF/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).