Metraipolis › archive › dc_f46c9fbe8beb
SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown — arXiv:2610.04008
- Source
- arXiv
- Original
- https://arxiv.org/abs/2610.04008
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T05:24:13.431Z
- SHA-256 of the archived text
ab759da7042b239d4517b092b20b77f5e0bc88865f8cef9f20e82f2548ebbb3a- Stable URL of this page
https://metraipolis.com/documento/dc_f46c9fbe8beb
How to cite
SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown — arXiv:2610.04008 — arXiv, https://arxiv.org/abs/2610.04008 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f46c9fbe8beb (Metraipolis archive dc_f46c9fbe8beb, sha256 ab759da7042b239d4517b092b20b77f5e0bc88865f8cef9f20e82f2548ebbb3a)
BibTeX
@misc{dcf46c9fbe8beb,
title = {SkillScriptBench: Benchmarking Self-Evolution of Executable Agent Skill Packages Beyond Markdown — arXiv:2610.04008},
howpublished = {arXiv},
url = {https://arxiv.org/abs/2610.04008},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f46c9fbe8beb; sha256 ab759da7042b239d4517b092b20b77f5e0bc88865f8cef9f20e82f2548ebbb3a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.04008. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Executable Agent Skills combine natural-language instructions and scripts into reusable packages for LLM agents, and revising them requires fixing errors without breaking correct behavior. Existing benchmarks do not systematically distinguish documentation repair, script repair, and preservation when evaluating skill self-evolution. We introduce SkillScriptBench, a 350-task benchmark designed to evaluate these capabilities separately. From a survey of over 35,000 GitHub-hosted Skill roots, we select 100 packages and construct 150 repair tasks. Each task pairs a package containing injected script faults with a maintenance request and executable checks of the required behavior. A complementary controlled track contains 200 tasks from 50 packages, each evaluated under the same maintenance request in four states: clean, documentation faults, script faults, and faults in both. Across four LLMs, methods that edit both documentation and scripts can repair script faults but do not consistently outperform Markdown-only revision on documentation repair or preservation. We therefore introduce AST-Guided Skill Revision, which uses abstract syntax trees and calling relationships to link maintenance requirements to relevant code locations. It restricts script edits to these locations and updates the documentation to match the revised scripts. Averaged across models, this revision stage yields absolute gains in repair success of 21.9% for Raw Package and 27.7% for CoEvoSkills on faulty packages. Absolute gains in the proportion of tasks solved in all three runs reach 20.8% and 31.5%, respectively, indicating more consistent repair success across repeated runs. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG); Software Engineering (cs.SE). Pagina del lavoro: https://arxiv.org/abs/2610.04008 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T05:24:05.415Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).