Metraipolis › archive › dc_b47dfeeb973e

PartHackBench: Certified Equal-Progress Stress Tests for Partial-Credit Tool-Agent Evaluation — arXiv:2609.29578

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.29578
License (as archived)
CC0-1.0
Archived
2026-10-08T04:26:16.093Z
SHA-256 of the archived text
b4275ff8662e68a32041f7f2767a0d0b8aded9d4f7e43db012d69201918e3567
Stable URL of this page
https://metraipolis.com/documento/dc_b47dfeeb973e

How to cite

PartHackBench: Certified Equal-Progress Stress Tests for Partial-Credit Tool-Agent Evaluation — arXiv:2609.29578 — arXiv (en), https://arxiv.org/abs/2609.29578 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b47dfeeb973e (Metraipolis archive dc_b47dfeeb973e, sha256 b4275ff8662e68a32041f7f2767a0d0b8aded9d4f7e43db012d69201918e3567)
BibTeX
@misc{dcb47dfeeb973e,
  title = {PartHackBench: Certified Equal-Progress Stress Tests for Partial-Credit Tool-Agent Evaluation — arXiv:2609.29578},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.29578},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b47dfeeb973e; sha256 b4275ff8662e68a32041f7f2767a0d0b8aded9d4f7e43db012d69201918e3567},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29578. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Long-horizon tool agents often make useful progress without reaching terminal success, motivating partial-credit evaluation. Yet evaluators may reward milestones that were temporary, later reversed, or not attributable to the evaluated agent. Comparing an honest trajectory with a higher-scoring adversarial one is inconclusive if the latter made more genuine progress. We introduce PartHackBench, a controlled methodology that removes this confound. A private certifier admits a pair only when its trajectories match component-wise in both current-state predicate satisfaction and standardized agent attribution; score inflation, defined as f(A) - f(H), is measured only afterward. In 18 sealed held-out tasks in PB-CSTE, the frozen historical-target run produced matched adversaries for 15 tasks. Historical credit yielded mean inflation of .252, conditional attack success of 10/15, end-to-end yield of 10/18, and detected none of 14 strict rollbacks. Semantic LLM judges were more resistant but remained vulnerable, especially under evaluator-targeted attacks, while PB-CSTE current-state controls, defined as exact functions of the certified components, yielded zero inflation by construction. PartHackBench thus provides a certified control for testing whether evaluator credit changes while all benchmark-defined task-relevant progress remains fixed.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-25 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Cryptography and Security (cs.CR); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.29578 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:57.801Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).