Metraipolis › archive › dc_6b59503666eb
The Cost of Context: Profiling the Energy Footprint of Input Tokens in Large Language Models — arXiv:2610.19819
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.19819
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T20:49:14.277Z
- SHA-256 of the archived text
588796d00065f7cacf2bccb04c27b2f307d599a2c68732b48f0f28f4240eb68f- Stable URL of this page
https://metraipolis.com/documento/dc_6b59503666eb
How to cite
The Cost of Context: Profiling the Energy Footprint of Input Tokens in Large Language Models — arXiv:2610.19819 — arXiv (en), https://arxiv.org/abs/2610.19819 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6b59503666eb (Metraipolis archive dc_6b59503666eb, sha256 588796d00065f7cacf2bccb04c27b2f307d599a2c68732b48f0f28f4240eb68f)
BibTeX
@misc{dc6b59503666eb,
title = {The Cost of Context: Profiling the Energy Footprint of Input Tokens in Large Language Models — arXiv:2610.19819},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.19819},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6b59503666eb; sha256 588796d00065f7cacf2bccb04c27b2f307d599a2c68732b48f0f28f4240eb68f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.19819. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The rapid adoption of Retrieval-Augmented Generation (RAG) and long-context LLMs has shifted the computational burden toward the prefill phase---the initial stage where the model processes all input tokens in parallel before generating a response---yet its energy impact remains poorly characterised. We investigate the contradiction between massive hardware-level power spikes and the sub-linear marginal cost of input tokens reported in system-level studies. Through high-resolution telemetry of Llama-3.1 and Qwen-2 models on NVIDIA A100 hardware, we identify the efficiency threshold where $O(n^2)$ attention complexity overcomes GPU parallelisation. Our results quantify how optimised memory orchestration (PagedAttention) delays this scaling bottleneck across three orders of context scaling, yielding up to an 8.9$\times$ energy efficiency advantage at 112,000 tokens compared to standard baselines. Furthermore, we characterise the energy disparity between input and output tokens, demonstrating that optimised serving (vLLM) causes this ratio to narrow (from $\approx$850$\times$ to 750$\times$) as context scales due to high decode efficiency, whereas unoptimised baselines show a much smaller, roughly constant gap ($\approx$150--230$\times$) because their prefill is itself expensive. This work establishes context-scaling laws that differentiate engine-level optimisations from inherent model constraints, providing an empirical framework for sustainable LLM deployment. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2610.19819 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T19:36:48.221Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).