Metraipolis › archive › dc_ec3b2ccbea13
Presentation Sensitivity in Typed Decision Models for Subjective Text Evaluation — arXiv:2611.06384
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.06384
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-05T18:18:46.172Z
- SHA-256 of the archived text
4e709d39bbe79a268250016c126a23f4e317820403fb18568e0deae06f868f97- Stable URL of this page
https://metraipolis.com/documento/dc_ec3b2ccbea13
How to cite
Presentation Sensitivity in Typed Decision Models for Subjective Text Evaluation — arXiv:2611.06384 — arXiv (en), https://arxiv.org/abs/2611.06384 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ec3b2ccbea13 (Metraipolis archive dc_ec3b2ccbea13, sha256 4e709d39bbe79a268250016c126a23f4e317820403fb18568e0deae06f868f97)
BibTeX
@misc{dcec3b2ccbea13,
title = {Presentation Sensitivity in Typed Decision Models for Subjective Text Evaluation — arXiv:2611.06384},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.06384},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ec3b2ccbea13; sha256 4e709d39bbe79a268250016c126a23f4e317820403fb18568e0deae06f868f97},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06384. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Typed decision models offer structured choices, scores, and probabilities for low-cost automated evaluation. We present a controlled audit of presentation sensitivity in Jev, Clef Flash, and Perplexity Decider across native choice and score interfaces. Across six datasets with 9,000 multiply annotated humor, helpfulness, and summarization items, we collect 702,000 judgments under interventions that vary positions, identifiers, authorship claims, endorsements, decision framing, and rating rubrics while preserving the evaluated texts. Jev's second-position probability advantage reaches 6.11 percentage points on summaries; reversing reader endorsements changes Perplexity's selected humor text on 40.40% of pairs; and rejection framing reduces Clef's summary agreement with human preferences by 23.36 points. Rating formats also shift score levels while often preserving rankings. Across these findings, repeatability, presentation robustness, and human alignment emerge as distinct properties of an evaluator. Reliable use therefore requires validating the decisions and score interpretations on which an evaluation pipeline depends. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.06384 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-05T18:13:03.944Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).