Metraipolis › archive › dc_0f52b045382a

Do Self-Evolving Skills Generalize to Held-Out Tasks? — arXiv:2609.39148

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39148
License (as archived)
CC0-1.0
Archived
2026-10-07T12:42:32.971Z
SHA-256 of the archived text
b1b9c4dc8fdf989498be40caea0b7a31ab0da268c0675ee34daadba3ae777b69
Stable URL of this page
https://metraipolis.com/documento/dc_0f52b045382a

How to cite

Do Self-Evolving Skills Generalize to Held-Out Tasks? — arXiv:2609.39148 — arXiv (en), https://arxiv.org/abs/2609.39148 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0f52b045382a (Metraipolis archive dc_0f52b045382a, sha256 b1b9c4dc8fdf989498be40caea0b7a31ab0da268c0675ee34daadba3ae777b69)
BibTeX
@misc{dc0f52b045382a,
  title = {Do Self-Evolving Skills Generalize to Held-Out Tasks? — arXiv:2609.39148},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39148},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0f52b045382a; sha256 b1b9c4dc8fdf989498be40caea0b7a31ab0da268c0675ee34daadba3ae777b69},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39148. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI agents can externalize what they learn from past tasks into reusable \emph{skills}, such as procedures, checklists, code, or other executable artifacts, that can be retrieved and reused when solving new tasks. Self-evolving skill methods keep rewriting these skills after each round of practice on training tasks, and the skill is then used on new tasks of the same kind. We ask a question: does the improvement a skill shows on its training tasks carry over to new test tasks? We test five self-evolving methods and a one-shot skill on six benchmarks, with the same model, the same agent, and the same train/test split for every method. Of the 21 skills that improve on their training tasks, 5 keep all of that improvement on the test tasks, 13 keep part of it, and 3 keep none of it. No existing method is best everywhere. When we read the skills, the ones that carry over badly often fix details that should depend on the task, such as column names and output files, or turn a fix for one failure into a rule for every task. An LLM judge that reads the skill content can often see this: it ranks finished skills the same way the test results do in 86\% of pairs. But it predicts the effect of a single edit poorly, so edits still have to be tested by running them. Based on these findings, we describe Generalizable Skill Optimization (GSO), which keeps only a guide for writing skills and writes a new skill for each task; it scores highest on all six benchmarks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.39148 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.738Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).