Metraipolis › archive › dc_18ffb2f27c4d

Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation — arXiv:2610.21135

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.21135
License (as archived)
CC0-1.0
Archived
2026-10-16T09:07:04.209Z
SHA-256 of the archived text
76af65dbc74e0b490fe819a88436096a74b49c357bf347047dacb75a29d4ed67
Stable URL of this page
https://metraipolis.com/documento/dc_18ffb2f27c4d

How to cite

Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation — arXiv:2610.21135 — arXiv (en), https://arxiv.org/abs/2610.21135 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_18ffb2f27c4d (Metraipolis archive dc_18ffb2f27c4d, sha256 76af65dbc74e0b490fe819a88436096a74b49c357bf347047dacb75a29d4ed67)
BibTeX
@misc{dc18ffb2f27c4d,
  title = {Stronger Judges Do Not See More: Deployment-Integrity Blindness in LLM Evaluation — arXiv:2610.21135},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21135},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_18ffb2f27c4d; sha256 76af65dbc74e0b490fe819a88436096a74b49c357bf347047dacb75a29d4ed67},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21135. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples, six tamper types, and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano. Stronger evaluators accept more tampered outputs: mean acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano), and a trigger-conditional tamper invisible on the evaluation suite produces 48-100% harmful outputs, rising with model capability. A PAC-style bound shows that tampers with zero output shift are undetectable from outputs at any sample size; a manifest-hash identity channel detects them with zero samples. Behavioral fingerprinting identifies the model component above chance (76.7%) but covers one of six stack components. We also propose a graded deployment-identity descriptor for evaluation reports.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.21135 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:17.910Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).