Metraipolis › archive › dc_fdf18443bc36
HERD: Harness-Evolving Recursive Distillation for Continual Learning — arXiv:2610.31401
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.31401
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-23T15:16:41.842Z
- SHA-256 of the archived text
c50149abc77e6c036fe6369e3d327dd63181d239d9027e647661c149ced44438- Stable URL of this page
https://metraipolis.com/documento/dc_fdf18443bc36
How to cite
HERD: Harness-Evolving Recursive Distillation for Continual Learning — arXiv:2610.31401 — arXiv (en), https://arxiv.org/abs/2610.31401 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fdf18443bc36 (Metraipolis archive dc_fdf18443bc36, sha256 c50149abc77e6c036fe6369e3d327dd63181d239d9027e647661c149ced44438)
BibTeX
@misc{dcfdf18443bc36,
title = {HERD: Harness-Evolving Recursive Distillation for Continual Learning — arXiv:2610.31401},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.31401},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fdf18443bc36; sha256 c50149abc77e6c036fe6369e3d327dd63181d239d9027e647661c149ced44438},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31401. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Continual learning requires a model to keep acquiring new skills and knowledge without compromising its existing capabilities. Parametric methods suffer from catastrophic forgetting and loss of plasticity. Non-parametric methods, such as prompts, skills, and harnesses, leverage in-context learning but are limited by context length and lossy plain-text compression. To effectively combine their complementary strengths, we propose HERD (Harness-Evolving Recursive Distillation), a hybrid continual learning framework that learns efficiently in the harness and internalizes experience into cumulative capabilities via distillation. On the harness side, HERD continually refines its skills to capture reusable patterns, while retaining high-residual examples that the skills fail to capture in a bounded memory. On the parameter side, HERD performs drift-constrained distillation with dual anchors, substantially reducing forgetting over long recursive runs. On the 25-stage task stream of TRACE+17, under fixed memory and compute budgets, HERD matches the strongest parametric baseline in new-task acquisition while exhibiting the least forgetting and the strongest retention of general capabilities. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.31401 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-23T15:09:08.617Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).