Metraipolis › archive › dc_f2aae101b2b7
IBBench-Light: A Paired Evaluation of Task-Conditioned Responses to External Directives — arXiv:2609.13725
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13725
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-15T22:47:18.023Z
- SHA-256 of the archived text
fbccd27085f4ddd52f0c95e5c6cb165cd7e4db0ff4febf08d88456f3d06241f4- Stable URL of this page
https://metraipolis.com/documento/dc_f2aae101b2b7
How to cite
IBBench-Light: A Paired Evaluation of Task-Conditioned Responses to External Directives — arXiv:2609.13725 — arXiv (en), https://arxiv.org/abs/2609.13725 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f2aae101b2b7 (Metraipolis archive dc_f2aae101b2b7, sha256 fbccd27085f4ddd52f0c95e5c6cb165cd7e4db0ff4febf08d88456f3d06241f4)
BibTeX
@misc{dcf2aae101b2b7,
title = {IBBench-Light: A Paired Evaluation of Task-Conditioned Responses to External Directives — arXiv:2609.13725},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13725},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f2aae101b2b7; sha256 fbccd27085f4ddd52f0c95e5c6cb165cd7e4db0ff4febf08d88456f3d06241f4},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13725. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
An external record may contain a procedure to apply or text to read, depending on the user's request. IBBench-Light tests both uses against the same record. Twelve semantic bases yield 144 matched pairs per model; four quantized instruction models produced 1,152 archived greedy responses. Paired exact-contract accuracy (PECA) requires both members to satisfy their output contracts. Qwen succeeds on 132 execute and 109 process prompts, but only 97 complete pairs, showing what marginal averages omit. We audit literal-target exposure and case normalization, then add 1,722 logged CPU generations to test directive-absent controls, twelve additional semantic bases, within-base wording changes, and generation stopping. In the pinned Phi rerun, changing the end-of-sequence (EOS) set changes exact paired success from 0/144 to 62/144. A bounded IHEval comparison uses the same SmolLM2 checkpoint and output budget while preserving its published instruction roles and scorer. The benchmark measures conditional task and output-contract success. Its task margins and paired count need to be read together with the stopping policy. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Logic in Computer Science (cs.LO). Pagina del lavoro: https://arxiv.org/abs/2609.13725 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T22:29:03.128Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).