Metraipolis › archive › dc_525d60abcd9f

Understanding the Energy Cost of Quantized Agents on Shared GPUs — arXiv:2610.41672

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41672
License (as archived)
CC0-1.0
Archived
2026-11-03T07:37:35.959Z
SHA-256 of the archived text
08de4a630743890ae53094ffdcc5b8a8758ce2f0bd9fdc1bd48ef62a0dc26416
Stable URL of this page
https://metraipolis.com/documento/dc_525d60abcd9f

How to cite

Understanding the Energy Cost of Quantized Agents on Shared GPUs — arXiv:2610.41672 — arXiv (en), https://arxiv.org/abs/2610.41672 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_525d60abcd9f (Metraipolis archive dc_525d60abcd9f, sha256 08de4a630743890ae53094ffdcc5b8a8758ce2f0bd9fdc1bd48ef62a0dc26416)
BibTeX
@misc{dc525d60abcd9f,
  title = {Understanding the Energy Cost of Quantized Agents on Shared GPUs — arXiv:2610.41672},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41672},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_525d60abcd9f; sha256 08de4a630743890ae53094ffdcc5b8a8758ce2f0bd9fdc1bd48ef62a0dc26416},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41672. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Quantization can lower the energy of language-model calls and free memory for serving more agents, but its benefit per completed task remains unclear when agents share a GPU. Measurements of fixed requests or isolated agents leave open how the agents' generated work interact with competition for compute and cache. We present a systematic study of precision and admission limits for two agent models on A100 deployments with 40 or 80 GB of memory, using coding and data-analysis tasks with independently verified outcomes. We replay recorded calls across precisions to isolate execution energy and cap the cache to test the value of freed memory. For both models with four admitted agents where requests fit in the cache, 4-bit weights with 16-bit activations execute the same calls with 10-25% less energy than 8-bit weights and activations. Despite this saving, extra work and fewer completions in live runs raise their energy per verified completion by 49%. Where requests compete for cache on the 40 GB deployment, 4-bit weights cost less, and capping their cache raises energy per verified completion by 34% with 16 agents. These results show how generated work and cache demand change the energy benefit of quantization, providing a practical basis for choosing precision and admission limits together.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.PF, nuovi lavori del giorno).
Categorie arXiv: Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2610.41672 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T07:30:32.593Z dalla pagina https://arxiv.org/list/cs.PF/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).