Metraipolis › archive › dc_53ee08d04e9e
What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute — arXiv:2610.02491
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.02491
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-05T22:48:58.817Z
- SHA-256 of the archived text
647035f4a1adeb093d3fe778e55186b5811a57bb8481af318918d3ec568eddb0- Stable URL of this page
https://metraipolis.com/documento/dc_53ee08d04e9e
How to cite
What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute — arXiv:2610.02491 — arXiv, https://arxiv.org/abs/2610.02491 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_53ee08d04e9e (Metraipolis archive dc_53ee08d04e9e, sha256 647035f4a1adeb093d3fe778e55186b5811a57bb8481af318918d3ec568eddb0)
BibTeX
@misc{dc53ee08d04e9e,
title = {What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute — arXiv:2610.02491},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.02491},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_53ee08d04e9e; sha256 647035f4a1adeb093d3fe778e55186b5811a57bb8481af318918d3ec568eddb0},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02491. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models spend the same amount of computation on every token they generate, regardless of how difficult each token is to produce. Methods such as speculative decoding and model routing are built on the premise that much of this computation is unnecessary, yet the computation an individual token actually requires has not been measured. We measure it through a Mixture-of-Agents (MoA) lens: a panel of fifteen language models of increasing capacity, drawn from three families, in which every agent attempts to reproduce a reference sequence token by token, conditioned on the correct preceding tokens. We define the inference cost of the smallest agent that succeeds as the token's sufficient compute, which upper-bounds what the token requires. On three core benchmarks, a 0.5B agent reproduces 92--95\% of reference tokens. Across Qwen, OLMo, and R1-distilled panels, the most expensive 10\% account for 64--80\% of estimated FLOPs. On all 500 MATH-500 problems, the MoA-derived map helps model routing reduce projected latency from 7.59 to 5.12 seconds while slightly improving accuracy, relative to the best confidence-routing baseline. The MoA-map helps drafting use 32.6\% fewer draft tokens and approximately 20\% lower projected latency than fixed-window drafting at similar accuracy. These comparisons reveal remaining allocation headroom, motivating controllers that exploit sufficient-compute structure. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.02491 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-05T22:45:55.586Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).