Metraipolis › archive › dc_676e8ecbfbd5

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses? — arXiv:2609.01600

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.01600
License (as archived): CC0-1.0
Archived: 2026-10-17T06:34:30.589Z
SHA-256 of the archived text: 116738d263c4ac72578bc91b8131f616a857385e3b4f1ddb2c14b7ac1ad7d50e
Stable URL of this page: https://metraipolis.com/documento/dc_676e8ecbfbd5

How to cite

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses? — arXiv:2609.01600 — arXiv (en), https://arxiv.org/abs/2609.01600 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_676e8ecbfbd5 (Metraipolis archive dc_676e8ecbfbd5, sha256 116738d263c4ac72578bc91b8131f616a857385e3b4f1ddb2c14b7ac1ad7d50e)

BibTeX

@misc{dc676e8ecbfbd5,
  title = {CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses? — arXiv:2609.01600},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.01600},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_676e8ecbfbd5; sha256 116738d263c4ac72578bc91b8131f616a857385e3b4f1ddb2c14b7ac1ad7d50e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.01600. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a local plugin change can propagate through dependencies and cleanup. We introduce CordisBench, a 1,200-question benchmark of this lifecycle reasoning. It combines a controlled formal setting with programs executed against Cordis, a runtime that manages component dependencies and cleanup, and asks models to identify affected components, predict state after a specified teardown order, determine which conditions hold under all or some orders, and choose reconfigurations that succeed when executed. Across these tasks, we evaluate three efficiency-oriented models at low reasoning effort with 2, 4, 8, 16, 24, or 32 relevant interactions, using deterministic task-specific scoring. Models usually handle small systems well but grow less reliable as more interactions become relevant, especially when predicting final state and when reasoning across teardown orders. Additional inference effort recovers marked gains for some models. The cost is nontrivial: on our 16-interaction subset, GPT-5.6 Luna uses nearly 3,000 reasoning tokens per question at medium effort. For these controlled instances, that cost is avoidable: an independent finite reference semantics agrees with Cordis execution on every observation and action outcome used for scoring across all 528 executable questions.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-02 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.01600 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T06:15:12.920Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).