Metraipolis › archive › dc_ae55e692e561
The SIMPLE Law for Energy Cost of LLM Inference — arXiv:2611.03330
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.03330
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:03:43.972Z
- SHA-256 of the archived text
098656770e4b4c292b33a5108682e76e99aa914d7b5b9278f64e6eb30e2902fd- Stable URL of this page
https://metraipolis.com/documento/dc_ae55e692e561
How to cite
The SIMPLE Law for Energy Cost of LLM Inference — arXiv:2611.03330 — arXiv (en), https://arxiv.org/abs/2611.03330 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ae55e692e561 (Metraipolis archive dc_ae55e692e561, sha256 098656770e4b4c292b33a5108682e76e99aa914d7b5b9278f64e6eb30e2902fd)
BibTeX
@misc{dcae55e692e561,
title = {The SIMPLE Law for Energy Cost of LLM Inference — arXiv:2611.03330},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.03330},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ae55e692e561; sha256 098656770e4b4c292b33a5108682e76e99aa914d7b5b9278f64e6eb30e2902fd},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03330. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Generative AI is billed per token, yet tokens are not equally expensive. Each token enters the KV-cache and raises the cost of those that follow, so energy grows non-linearly along a sequence whose tokens look identical. No analytical model states what a token costs, and the most accurate SOTA model is a fit, not a law: it absorbs architecture into fitted coefficients, requiring per-LLM fitting. We derive closed-form expressions of the computational and memory-access complexity of LLM inference in terms of architectural hyperparameters, covering MHA and GQA, and compare a family of analytical models of increasing granularity. From these, we obtain the SIMPLE Inference Model to Predict LLMs Energy (SIMPLE), a four-parameter law that separates prefill from decode, and decode into KV-cache cost and next-token computation. Over traces of 13 dense LLMs from six families on NVIDIA H100 GPUs, it attains R2 = 0.969 and 11.97% MAPE, 3.1-3.5x below the SOTA, without per-LLM calibration. The SIMPLE law explains as well as predicts: prefill energy grows quadratically with the hidden dimension, so larger LLMs suffer most from long inputs; and more aggressive GQA KV grouping reduces the per-token energy by up to 3x relative to standard MHA. Increasing the reasoning token budget pushes output length beyond the analytically derived energy minimum, driving super-linear energy growth; although accuracy grows with the budget, energy grows faster, so Intelligence per Joule peaks at an intermediate budget and falls by up to 3.14x across a 4.0x reasoning-budget range. Agentic serving raises energy per request by 13.0x relative to non-agentic ones, driven by the growth of both the prefill and the KV-cache terms with input length; while prefix caching removes most of the prefill cost (-93%), the KV-cache term is unaffected, so even a cached agentic request costs 4.8x more. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2611.03330 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T10:59:45.886Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).