Metraipolis › archive › dc_aed73094ee78

Predictable performance is suboptimal, and what it means for model routing — arXiv:2610.31428

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.31428
License (as archived): CC0-1.0
Archived: 2026-10-25T08:35:36.925Z
SHA-256 of the archived text: 31229abca719af21403104316f491777869ce8a3ad04d166d448ddd60088fb94
Stable URL of this page: https://metraipolis.com/documento/dc_aed73094ee78

How to cite

Predictable performance is suboptimal, and what it means for model routing — arXiv:2610.31428 — arXiv (en), https://arxiv.org/abs/2610.31428 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aed73094ee78 (Metraipolis archive dc_aed73094ee78, sha256 31229abca719af21403104316f491777869ce8a3ad04d166d448ddd60088fb94)

BibTeX

@misc{dcaed73094ee78,
  title = {Predictable performance is suboptimal, and what it means for model routing — arXiv:2610.31428},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.31428},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aed73094ee78; sha256 31229abca719af21403104316f491777869ce8a3ad04d166d448ddd60088fb94},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31428. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) differ in capabilities and costs, motivating routing methods that adaptively choose a model for each query. These methods rely on performance prediction: given a query, estimate whether a model will answer it correctly. Most routing methods train separate predictors of candidate models' performance to guide model selection. Our work shows that performance prediction only beats a simple baseline when the candidate model is suboptimal on the task or the performance predictor adds significant capacity relative to the candidate model. Specifically, we take a model's self-confidence, the probability it assigns to its chosen answer, as a baseline and study how much a performance predictor can improve on it. First, we show that any performance predictor that outperforms a model's self-confidence can be used to improve the model's loss: the predictor's advantage transfers exactly in log loss through recalibration. Second, accounting for the predictor's additional parameters, we bound its advantage by the model's suboptimality plus the gain from those parameters. Third, we extend our theoretical results to two-model cascades, which route queries to a more capable model when the first model is predicted to fail. Experiments with 16 language models on four benchmarks show that predictors gain a larger advantage over self-confidence on more suboptimal models. This advantage shrinks as models improve through finetuning. Cascading exhibits the same two patterns: predictor-based routing offers larger gains over confidence-based routing for more suboptimal models, and these gains diminish after finetuning.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.31428 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T08:31:13.267Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).