Metraipolis › archive › dc_989e8be9196d

The SIMPLE Law for Energy Cost of LLM Inference — arXiv:2611.03330

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03330
License (as archived)
CC0-1.0
Archived
2026-11-03T10:24:15.385Z
SHA-256 of the archived text
4d4b45a22c1c61e47e8e8a2921d46384d053f961daa4b3ce561251790da80c5f
Stable URL of this page
https://metraipolis.com/documento/dc_989e8be9196d

How to cite

The SIMPLE Law for Energy Cost of LLM Inference — arXiv:2611.03330 — arXiv (en), https://arxiv.org/abs/2611.03330 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_989e8be9196d (Metraipolis archive dc_989e8be9196d, sha256 4d4b45a22c1c61e47e8e8a2921d46384d053f961daa4b3ce561251790da80c5f)
BibTeX
@misc{dc989e8be9196d,
  title = {The SIMPLE Law for Energy Cost of LLM Inference — arXiv:2611.03330},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03330},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_989e8be9196d; sha256 4d4b45a22c1c61e47e8e8a2921d46384d053f961daa4b3ce561251790da80c5f},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03330. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generative AI is billed per token, yet tokens are not equally expensive. Each token enters the KV-cache and raises the cost of those that follow, so energy grows non-linearly along a sequence whose tokens look identical. No analytical model states what a token costs, and the most accurate SOTA model is a fit, not a law: it absorbs architecture into fitted coefficients, requiring per-LLM fitting. We derive closed-form expressions of the computational and memory-access complexity of LLM inference in terms of architectural hyperparameters, covering MHA and GQA, and compare a family of analytical models of increasing granularity. From these, we obtain the SIMPLE Inference Model to Predict LLMs Energy (SIMPLE), a four-parameter law that separates prefill from decode, and decode into KV-cache cost and next-token computation. Over traces of 13 dense LLMs from six families on NVIDIA H100 GPUs, it attains R2 = 0.969 and 11.97% MAPE, 3.1-3.5x below the SOTA, without per-LLM calibration. The SIMPLE law explains as well as predicts: prefill energy grows quadratically with the hidden dimension, so larger LLMs suffer most from long inputs; and more aggressive GQA KV grouping reduces the per-token energy by up to 3x relative to standard MHA. Increasing the reasoning token budget pushes output length beyond the analytically derived energy minimum, driving super-linear energy growth; although accuracy grows with the budget, energy grows faster, so Intelligence per Joule peaks at an intermediate budget and falls by up to 3.14x across a 4.0x reasoning-budget range. Agentic serving raises energy per request by 13.0x relative to non-agentic ones, driven by the growth of both the prefill and the KV-cache terms with input length; while prefix caching removes most of the prefill cost (-93%), the KV-cache term is unaffected, so even a cached agentic request costs 4.8x more.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2611.03330 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T10:19:49.945Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).