Metraipolis › archive › dc_2bb590fdcd98

Evidence Lock Before Commitment: A Frozen Interface Degrades LLM-as-Judge Evaluation — arXiv:2608.05353

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.05353
License (as archived): CC0-1.0
Archived: 2026-10-25T21:27:06.841Z
SHA-256 of the archived text: 540645886ea23408fa388e5ce575c3a92d724633b7449a57b5d7959ec2707b4c
Stable URL of this page: https://metraipolis.com/documento/dc_2bb590fdcd98

How to cite

Evidence Lock Before Commitment: A Frozen Interface Degrades LLM-as-Judge Evaluation — arXiv:2608.05353 — arXiv (en), https://arxiv.org/abs/2608.05353 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2bb590fdcd98 (Metraipolis archive dc_2bb590fdcd98, sha256 540645886ea23408fa388e5ce575c3a92d724633b7449a57b5d7959ec2707b4c)

BibTeX

@misc{dc2bb590fdcd98,
  title = {Evidence Lock Before Commitment: A Frozen Interface Degrades LLM-as-Judge Evaluation — arXiv:2608.05353},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.05353},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2bb590fdcd98; sha256 540645886ea23408fa388e5ce575c3a92d724633b7449a57b5d7959ec2707b4c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.05353. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM judges are often asked to extract criteria and evidence before choosing between candidate answers. This workflow assumes that the intermediate record preserves the information needed for a later verdict. For reasoning-capable models, visible field order does not reveal internal decision order, so we test an observable alternative: persist the evidence in one call and make it the exclusive input to the next. Across 24,000 judgments over HelpSteer3, FeedbackQA, and CoVal, we compare standard pairwise judging, structured one-call judging, two-call evidence locking, and three-call pointwise locking with Claude Sonnet 4.5 and GPT-5. Evidence locking reduces agreement with released human preferences by 4 to 6 percentage points and increases answer-order inconsistency by 8 to 10 points relative to structured one-call judging. Pointwise locking is also harmful, while structured evidence elicitation remains close to standard judging. The result holds for both judges and all three datasets. Persisted evidence can support auditability, but it should not replace the source answers at decision time.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-07 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.05353 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-25T21:24:38.187Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).