Metraipolis › archive › dc_a2940694bb25
Position Bias Undermines Preference Consistency in Listwise LLM-Based Reranking — arXiv:2608.03091
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.03091
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-31T12:57:52.206Z
- SHA-256 of the archived text
6dfebd71950f41cf5e14bbb6a8d532895b8fb941b23b4c006523fbab45d1db94- Stable URL of this page
https://metraipolis.com/documento/dc_a2940694bb25
How to cite
Position Bias Undermines Preference Consistency in Listwise LLM-Based Reranking — arXiv:2608.03091 — arXiv (en), https://arxiv.org/abs/2608.03091 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a2940694bb25 (Metraipolis archive dc_a2940694bb25, sha256 6dfebd71950f41cf5e14bbb6a8d532895b8fb941b23b4c006523fbab45d1db94)
BibTeX
@misc{dca2940694bb25,
title = {Position Bias Undermines Preference Consistency in Listwise LLM-Based Reranking — arXiv:2608.03091},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.03091},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a2940694bb25; sha256 6dfebd71950f41cf5e14bbb6a8d532895b8fb941b23b4c006523fbab45d1db94},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.03091. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large language models (LLMs) have emerged as promising listwise rerankers for recommender systems, but their reliability under equivalent candidate permutations remains unclear. Since recommendation candidates form an unordered set, a reranker should not depend on the arbitrary order used to serialize them. However, decoder-only LLM rerankers can allow input order to affect model scores, pairwise preferences, and rankings. We study how position bias affects the ranking process induced by LLM-based rerankers. Instead of measuring only changes in final ranked lists, we treat rankings produced under equivalent candidate permutations as observations of an induced preference system. We introduce an evaluation framework measuring pairwise preference instability, global preference inconsistency, and listwise output consistency. This framework characterizes candidate-order sensitivity at the pairwise, global, and output levels. Experiments across multiple LLMs, datasets, and list lengths show that these consistency measures are closely aligned, but can diverge from recommendation effectiveness and marginal position-exposure bias. Improving relevance or flattening exposure across positions does not necessarily restore stable pairwise preferences, globally coherent preference structures, or consistent ranked outputs. These results show that reducing marginal exposure skew is insufficient to establish ranking-function validity in LLM-based reranking. Code is available at this https URL . Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-05 in cs.IR). Categorie arXiv: Information Retrieval (cs.IR). Pagina del lavoro: https://arxiv.org/abs/2608.03091 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-31T12:45:13.293Z dalla pagina https://arxiv.org/catchup/cs.IR/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).