Metraipolis › archive › dc_7efca8c7d169

Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It — arXiv:2609.26758

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.26758
License (as archived)
CC0-1.0
Archived
2026-10-08T13:45:50.982Z
SHA-256 of the archived text
d703f078018aab34e022e62236bcf46b4238b2d9c225387aa46e9398fc8d2433
Stable URL of this page
https://metraipolis.com/documento/dc_7efca8c7d169

How to cite

Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It — arXiv:2609.26758 — arXiv (en), https://arxiv.org/abs/2609.26758 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7efca8c7d169 (Metraipolis archive dc_7efca8c7d169, sha256 d703f078018aab34e022e62236bcf46b4238b2d9c225387aa46e9398fc8d2433)
BibTeX
@misc{dc7efca8c7d169,
  title = {Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It — arXiv:2609.26758},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.26758},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7efca8c7d169; sha256 d703f078018aab34e022e62236bcf46b4238b2d9c225387aa46e9398fc8d2433},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26758. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Typed decision models are built for settings where model outputs are consumed directly by software. Instead of generating free-form text, they return a decision over a predefined set of options. By construction, every output conforms to the required schema. Yet this guarantee does not tell us whether the model interprets the options as intended. We study Jev and two Jev-like models with open weights by changing how option names are assigned to rubrics. Each option consists of an option name and a textual rubric that defines what the option means. We change only which option name is assigned to each rubric; the question, state, rubric wording, and set of option names remain exactly the same. On 1200 workflow decisions with task-specific rubrics, renaming the two options from 0/1 to no/yes changes 70.4 more answers per hundred (95% CI: [67.6, 73.1]) and shifts AUC from .94 to .23, revealing a systematic reversal in the decision ranking rather than simple uncertainty. The same operation has little effect with neutral option names. This pattern holds across all 4 predicates, where the effect is at least 7.4x larger than under the neutral control, and becomes stronger as the number of options increases. The effect also depends on the read-out geometry: a second model family that mean-pools over the full option span flips 4.1x less often. The hosted model exhibits the same behavior: the swap changes AUC from .8146 to .5806 and produces 24x as many answer flips as its test-retest floor. In contrast, replacing the option names with random character strings returns all model families to the neutral-control regime without reducing accuracy. The failure therefore depends on the semantic polarity of the option names rather than on the renaming operation itself. Across all conditions, the type-error rate remains 0%, even when decision accuracy degrades substantially.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.26758 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T13:18:38.177Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).