Metraipolis › archive › dc_6698d04bb0de

EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness? — arXiv:2609.04280

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.04280
License (as archived)
CC0-1.0
Archived
2026-10-16T22:18:16.664Z
SHA-256 of the archived text
24c8301a76884f579dbec63d77678471b90cc5e9aa78719e48783598b86491ec
Stable URL of this page
https://metraipolis.com/documento/dc_6698d04bb0de

How to cite

EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness? — arXiv:2609.04280 — arXiv (en), https://arxiv.org/abs/2609.04280 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6698d04bb0de (Metraipolis archive dc_6698d04bb0de, sha256 24c8301a76884f579dbec63d77678471b90cc5e9aa78719e48783598b86491ec)
BibTeX
@misc{dc6698d04bb0de,
  title = {EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness? — arXiv:2609.04280},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.04280},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6698d04bb0de; sha256 24c8301a76884f579dbec63d77678471b90cc5e9aa78719e48783598b86491ec},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.04280. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Modern LLM-based agents operate through a harness of tools, reusable skills, and specialist agents that shapes what they observe and what they can do. In practice, this harness continually evolves as new capabilities are added. We introduce EvoHarnessBench, a benchmark for evaluating agents under controlled harness evolution across three axes (tools, skills, and agents). Unlike existing continual-learning benchmarks for agents, which typically place non-stationarity (i.e., what changes over time) in the task stream while keeping the harness fixed, EvoHarnessBench places non-stationarity in the externally supplied harness itself. It contains 17 multi-stage harness streams constructed deterministically from verifier-based benchmarks, comprising 802 tasks, 520 tools, 42 skills, and 62 agents. We evaluate two complementary settings corresponding to the central challenges of harness evolution: deployment evaluation, which isolates retention of previously accessible competence as the harness expands, and self-evolving adaptation evaluation, which tests whether accumulated experience remains useful as new capabilities are introduced. Our results reveal three persistent gaps. First, harness expansion alone can degrade performance on previously solved tasks, producing harness-induced forgetting. Second, gains from self-evolving adaptation remain inconsistent across stages of harness evolution, capability axes, and environments. Third, retention and adaptation can pull in different directions: preserving earlier competence does not necessarily improve adaptation to newly introduced capabilities, and vice versa. These results establish harness evolution as a distinct challenge for building agents that can keep pace with an evolving harness while preserving previously effective behavior.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-07 in cs.MA).
Categorie arXiv: Multiagent Systems (cs.MA) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.04280 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T22:09:18.990Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-09-07?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).