Metraipolis › archive › dc_ad84b8a18176

$Δτ$-Bench: Pushing Agents Toward a Moving Capability — arXiv:2610.34238

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.34238
License (as archived)
CC0-1.0
Archived
2026-10-27T08:01:13.390Z
SHA-256 of the archived text
529ff6fc8bbd39fbc857eede9457d37bf3fbe48b36b20cb6a505a2d5368a6bdc
Stable URL of this page
https://metraipolis.com/documento/dc_ad84b8a18176

How to cite

$Δτ$-Bench: Pushing Agents Toward a Moving Capability — arXiv:2610.34238 — arXiv (en), https://arxiv.org/abs/2610.34238 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ad84b8a18176 (Metraipolis archive dc_ad84b8a18176, sha256 529ff6fc8bbd39fbc857eede9457d37bf3fbe48b36b20cb6a505a2d5368a6bdc)
BibTeX
@misc{dcad84b8a18176,
  title = {$Δτ$-Bench: Pushing Agents Toward a Moving Capability — arXiv:2610.34238},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.34238},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ad84b8a18176; sha256 529ff6fc8bbd39fbc857eede9457d37bf3fbe48b36b20cb6a505a2d5368a6bdc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34238. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In this paper, we introduce $\Delta\tau$-Bench, an executable agent benchmark that models the structural evolution of large tool ecosystems. In $\Delta\tau$-Bench, independently evolving services continually reshape how tasks can be accomplished: familiar tools may disappear or be replaced, while more efficient alternatives emerge amid overlapping functionality and noisy documentation. Against this moving capability frontier, we evaluate whether agents can proactively discover efficient tool-use paths and promptly revise their accumulated procedural knowledge, bringing together demands that existing benchmarks fail to examine. $\Delta\tau$-Bench comprises 1,588 validated multi-turn tasks across 42 domains and six successive ecosystem states generated through capability-preserving transformations. Our evaluation shows that even the strongest evaluated model or harness achieves only 42.3% task success, while initial gains from existing skill-learning methods can not consistently translate into better adaptation as the ecosystem evolves. These findings motivate Steward, an ecosystem manager that jointly manages tools and procedural knowledge through incremental skill maintenance and dynamic provisioning during task execution. Experiments show that Steward sustains higher task success across ecosystem changes with lower skill-maintenance overhead, while improving both recovery from disrupted workflows and adoption of more efficient alternatives.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR); Machine Learning (cs.LG); Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2610.34238 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-27T07:55:30.885Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).