Metraipolis › archive › dc_8192edca3cb1

Does a model's stated reason for rejecting a candidate do any work? — arXiv:2609.30151

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.30151
License (as archived)
CC0-1.0
Archived
2026-10-08T04:28:02.740Z
SHA-256 of the archived text
5b9fd31a397fcb78cd0e3bc2c80fc33523ae1cd80d3962250067f49f8bed4f64
Stable URL of this page
https://metraipolis.com/documento/dc_8192edca3cb1

How to cite

Does a model's stated reason for rejecting a candidate do any work? — arXiv:2609.30151 — arXiv (en), https://arxiv.org/abs/2609.30151 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8192edca3cb1 (Metraipolis archive dc_8192edca3cb1, sha256 5b9fd31a397fcb78cd0e3bc2c80fc33523ae1cd80d3962250067f49f8bed4f64)
BibTeX
@misc{dc8192edca3cb1,
  title = {Does a model's stated reason for rejecting a candidate do any work? — arXiv:2609.30151},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.30151},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8192edca3cb1; sha256 5b9fd31a397fcb78cd0e3bc2c80fc33523ae1cd80d3962250067f49f8bed4f64},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.30151. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Asked to choose between candidates and explain the choice, a language model often rejects a rival by naming a fact its profile lacks: no director, no date of death. That sentence is a claim about the text in front of the model, and it can be tested without any judge. We insert a real corpus sentence stating the named fact into the rival's profile and ask again under greedy decoding. Two controls separate content from placement: a length-matched irrelevant sentence at the same profile, and the same two sentences at a third option the model never mentioned. In the largest of three runs, six open models on 2WikiMultihopQA, supplying the named fact at the profile the model named moves its choice more than the irrelevant control does, odds ratio 3.57 [1.54, 8.26], Holm p=0.0210, and this survives dropping any single model. The contrast the design was built to detect, the same fact at the option nobody named, does not clear correction, Holm p=0.2428. The strongest result in the family carries no content claim at all: the identical irrelevant sentence moves the choice more at the named rival than at the third option, Holm p=0.0008. Repair and control also differ in co-candidate mentions, relation template and fluency; post-hoc matching on the first two preserves the content effects' direction, matching fluency weakens one, so the content contrasts bound an effect rather than establish one. A forced single-token probability read disagrees in direction with the free-text choice on that same contrast, and three candidate explanations for the disagreement find no support. Every measurement is a string rule, so each was validated against the records it reads; validation caught eight defects. The largest, a choice-parsing rule that returned the option a model had just rejected in 17.1% of adjudicable responses, would have reported six surviving contrasts instead of four.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.30151 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:58.242Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).