Metraipolis › archive › dc_a488ca9a0f35
Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill — arXiv:2611.09028
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.09028
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:33:51.709Z
- SHA-256 of the archived text
b6481746ad07897401b509fa69f84a8dbcbc8982f52c7fe06b289a4e7d4d5807- Stable URL of this page
https://metraipolis.com/documento/dc_a488ca9a0f35
How to cite
Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill — arXiv:2611.09028 — arXiv (en), https://arxiv.org/abs/2611.09028 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a488ca9a0f35 (Metraipolis archive dc_a488ca9a0f35, sha256 b6481746ad07897401b509fa69f84a8dbcbc8982f52c7fe06b289a4e7d4d5807)
BibTeX
@misc{dca488ca9a0f35,
title = {Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill — arXiv:2611.09028},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.09028},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a488ca9a0f35; sha256 b6481746ad07897401b509fa69f84a8dbcbc8982f52c7fe06b289a4e7d4d5807},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09028. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Routers typically choose a model per query by trading predicted correctness against cost. Most price each model at a fixed cost, as if every query used the same number of output tokens, but the output length of reasoning models varies by an order of magnitude across queries. When is per-query cost worth predicting? What factors drive cost? How cheaply can predictions be made? We predict both the success probability and the output length of candidate models from the prefill activations of a single small frozen LLM, the scout. With reasoning candidates, perfect per-query cost knowledge would save 29-61% of spending on eight of nine datasets, and much less with non-reasoning candidates on the same problems. On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing, up to 69% of what perfect knowledge would save. They are significantly cheaper than each estimator we reimplemented or adapted from published work, including ZeroRouter and a MixLLM-style predictor, on a majority of datasets. Output length depends mostly on the problem. On most math and code datasets, harder problems are longer, so predicted success alone prices cost nearly as well. On knowledge and mixed-format datasets they are not, and predicted success falls well behind a probe trained on length. The probe keeps most of its saving with 20 labelled problems and transfers without labels to four of seven new benchmarks. On a mixture of all nine benchmarks, it saves 22% beyond pricing each benchmark at its own median length. Savings stop growing at about a 2B scout, so above 4B a larger scout costs more than it adds. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2611.09028 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:30:47.263Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).