Metraipolis › archive › dc_f5e273658c7f

Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference — arXiv:2611.10193

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10193
License (as archived)
CC0-1.0
Archived
2026-11-10T18:40:48.350Z
SHA-256 of the archived text
4724fcb738844d209990fa86ee67b06d2d4a92edfc0e6502cf7d9566f051993e
Stable URL of this page
https://metraipolis.com/documento/dc_f5e273658c7f

How to cite

Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference — arXiv:2611.10193 — arXiv (en), https://arxiv.org/abs/2611.10193 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f5e273658c7f (Metraipolis archive dc_f5e273658c7f, sha256 4724fcb738844d209990fa86ee67b06d2d4a92edfc0e6502cf7d9566f051993e)
BibTeX
@misc{dcf5e273658c7f,
  title = {Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference — arXiv:2611.10193},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10193},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f5e273658c7f; sha256 4724fcb738844d209990fa86ee67b06d2d4a92edfc0e6502cf7d9566f051993e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10193. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) increasingly support Web-based services that must balance response quality and inference cost across heterogeneous requests. Conventional routing selects one model per prompt, while fixed ensembles cannot adapt their composition and cost to individual requests. We introduce PISA, a prompt-adaptive, cost-aware single-model router, and PISA-E, its extension to ensemble routing. At the core of both methods, PISA combines soft cluster memberships with model-specific predictors to estimate individual model capabilities. Building on these estimates, PISA-E jointly selects ensemble composition and cardinality under an explicit cost--quality preference that can be adjusted at inference time without retraining. A tractable ensemble-success surrogate and efficient candidate selection enable this extension without ensemble-specific training. Across four LLM routing benchmarks under in-distribution (ID) and out-of-distribution (OOD) conditions, PISA achieves the highest macro-average cost--performance hypervolume among the evaluated single-model routers in both settings. Meanwhile, PISA-E achieves the highest macro-average task performance among the evaluated ensemble approaches when prioritizing response quality, surpassing the strongest competitors by 2.09 and 2.87 percentage points under ID and OOD conditions, respectively. At selected lower-cost operating points, it also exceeds the strongest ensemble competitors' macro-average task performance while reducing average inference cost by approximately 36% and 25%, respectively. These results demonstrate the effectiveness of prompt-adaptive capability estimation for both single-model routing and selective ensembling.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.10193 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.482Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).