Metraipolis › archive › dc_485d0f02ec85
Creating Power Flexibility in Datacenters via Query Routing — arXiv:2611.05148
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.05148
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T21:38:16.193Z
- SHA-256 of the archived text
b92912ac34a34d91dc2b536c6ea67f10d6f40c24c3263b778cb60ec4b1b1749e- Stable URL of this page
https://metraipolis.com/documento/dc_485d0f02ec85
How to cite
Creating Power Flexibility in Datacenters via Query Routing — arXiv:2611.05148 — arXiv (en), https://arxiv.org/abs/2611.05148 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_485d0f02ec85 (Metraipolis archive dc_485d0f02ec85, sha256 b92912ac34a34d91dc2b536c6ea67f10d6f40c24c3263b778cb60ec4b1b1749e)
BibTeX
@misc{dc485d0f02ec85,
title = {Creating Power Flexibility in Datacenters via Query Routing — arXiv:2611.05148},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.05148},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_485d0f02ec85; sha256 b92912ac34a34d91dc2b536c6ea67f10d6f40c24c3263b778cb60ec4b1b1749e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.05148. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The growing energy requirements of large language model inference highlight the need for datacenters to support safe grid operation by providing power demand flexibility, while maintaining service for interactive workloads. This work investigates query routing as a mechanism to enable such capability by dynamically allocating queries across different language model instances without deferring requests. We develop a general analytical framework that connects query arrival rates and sequence length distributions to token iteration time, request residence time, serving capacity, and power consumption of model instances. Leveraging this framework, we characterize the feasible query routing allocations and the corresponding power demand flexibilities, enabling the selection of routing strategies to achieve desired power consumption behavior. We validate our framework across four GPU platforms on language models ranging from 1B to 70B parameters, obtaining 2.8% mean absolute percentage error in estimating achievable power flexibility. We evaluate our approach on a real trace-driven simulation of a 10,000 H100 GPUs datacenter, exhibiting flexibility up to 16% of total power consumption, with 90% of the desired power reduction achieved within approximately 10 seconds. These results establish query routing as an effective mechanism for datacenter power demand flexibility and demonstrate the broader applicability of our proposed framework to datacenter level planning and orchestration. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2611.05148 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T21:35:49.586Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).