Metraipolis › archive › dc_0be8b67ffa68
System One models Are Not Better Rerankers — arXiv:2610.32318
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.32318
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T19:53:35.354Z
- SHA-256 of the archived text
6787c88c565587760f46b2179a58acb5af7d2221b00ef7e18ce880a4d92bf772- Stable URL of this page
https://metraipolis.com/documento/dc_0be8b67ffa68
How to cite
System One models Are Not Better Rerankers — arXiv:2610.32318 — arXiv (en), https://arxiv.org/abs/2610.32318 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0be8b67ffa68 (Metraipolis archive dc_0be8b67ffa68, sha256 6787c88c565587760f46b2179a58acb5af7d2221b00ef7e18ce880a4d92bf772)
BibTeX
@misc{dc0be8b67ffa68,
title = {System One models Are Not Better Rerankers — arXiv:2610.32318},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.32318},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0be8b67ffa68; sha256 6787c88c565587760f46b2179a58acb5af7d2221b00ef7e18ce880a4d92bf772},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32318. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
System One models, or decision models, answer a question about an input with a calibrated probability instead of generated text. Within weeks of the first release, in September 2026, they were used as rerankers. We test whether the decision fine-tuning behind them makes a better reranker than plain reranker fine-tuning of the same model. For six Qwen backbones from 0.6B to 27B parameters and ten TREC DL and BEIR collections, we compare the untuned model, asked zero-shot, with its decision fine-tunes and its reranker fine-tunes, two of which we train ourselves. In our work, we draw the conclusion that decision fine-tuning never makes the better reranker. From 0.6B to 9B, reranker fine-tuning wins on every backbone by 5 to 29 points nDCG@10, also on zero-shot collections, and at 27B the two tie. Up to 9B, decision fine-tuning adds nothing over simply asking the untuned model. Moreover, we find that the calibration advantage of decision models depends on the question. Asked whether a passage answers the query, they are better calibrated than the published reranker fine-tunes from 4B on. Asked a slightly different question from prior work, they lose this advantage. Lastly, calibration does not make up for a worse ranking: up to 9B, a reranker fine-tune keeps the better passages at a fixed threshold. If you need a reranker, fine-tune a reranker. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2610.32318 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-26T19:45:48.356Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).