Metraipolis › archive › dc_6b25ea3ca852
Which LLM to pick? Online Active Model Selection for Large Language Models — arXiv:2610.01592
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.01592
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T13:55:48.575Z
- SHA-256 of the archived text
f3e844bbdae8ba6c08416756b5ea0801569504efab457198b6f363dee67e047e- Stable URL of this page
https://metraipolis.com/documento/dc_6b25ea3ca852
How to cite
Which LLM to pick? Online Active Model Selection for Large Language Models — arXiv:2610.01592 — arXiv (en), https://arxiv.org/abs/2610.01592 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6b25ea3ca852 (Metraipolis archive dc_6b25ea3ca852, sha256 f3e844bbdae8ba6c08416756b5ea0801569504efab457198b6f363dee67e047e)
BibTeX
@misc{dc6b25ea3ca852,
title = {Which LLM to pick? Online Active Model Selection for Large Language Models — arXiv:2610.01592},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.01592},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6b25ea3ca852; sha256 f3e844bbdae8ba6c08416756b5ea0801569504efab457198b6f363dee67e047e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.01592. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models (LLMs) are increasingly applied to process streaming data, with practitioners relying on benchmarks to select the best model even though these signals only approximate real performance. While oracle annotations can provide reliable feedback, they are often costly and difficult to obtain at scale. To address this challenge, we propose ONLINE LLM PICKER, the first framework for active model selection for LLMs in online settings. Given an arbitrary stream of queries and a limited annotation budget, ONLINE LLM PICKER selects the most informative prompts for annotation to identify the best LLM among candidate models. Across multiple tasks including 10 datasets, for over 130 language models, we show that ONLINE LLM PICKER saves annotation cost by up to 71.67% while reliably identifying the best or near-best model for the stream. We also show that using the returned model for sequential generation on unannotated prompts across the stream reduces regret by up to a factor of 2.51x, indicating that ONLINE LLM PICKER can identify the best or near-best model well before processing all streaming prompts. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.CL). Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.01592 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T12:15:34.733Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).