Metraipolis › archive › dc_2fd377df3fa5

Harness-Aware Distillation for Small Language Model Agents — arXiv:2610.02858

Source: arXiv
Original: https://arxiv.org/abs/2610.02858
License (as archived): CC0-1.0
Archived: 2026-10-05T22:49:19.231Z
SHA-256 of the archived text: d1983833a6b141db31db54a11141a96204a9f8b24ae001355cf4eed7904704f6
Stable URL of this page: https://metraipolis.com/documento/dc_2fd377df3fa5

How to cite

Harness-Aware Distillation for Small Language Model Agents — arXiv:2610.02858 — arXiv, https://arxiv.org/abs/2610.02858 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2fd377df3fa5 (Metraipolis archive dc_2fd377df3fa5, sha256 d1983833a6b141db31db54a11141a96204a9f8b24ae001355cf4eed7904704f6)

BibTeX

@misc{dc2fd377df3fa5,
  title = {Harness-Aware Distillation for Small Language Model Agents — arXiv:2610.02858},
  howpublished = {arXiv},
  url = {https://arxiv.org/abs/2610.02858},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2fd377df3fa5; sha256 d1983833a6b141db31db54a11141a96204a9f8b24ae001355cf4eed7904704f6},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv, https://arxiv.org/abs/2610.02858. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Language model agents are deployed with a harness, the software around the model that manages its context, tools, and feedback. When such an agent is distilled into a smaller one, the harness stays in place, so the student mainly needs the teacher-specific abilities that the harness cannot provide, such as acting correctly on harness information. Standard distillation, however, imitates the teacher's full outputs and treats the harness as part of the input. We propose Harness-Aware Distillation (HAD), which focuses distillation on what the teacher adds beyond the harness. HAD complements on-policy distillation with two components: an action preference that contrasts the same teacher's actions with and without the harness information, scored after the student's own reasoning, and a validity check that drops preference pairs whose preferred action contradicts the harness records. We show that the contrast gives the student information that imitating the teacher alone cannot provide, and HAD needs no task rewards, success labels, or future information. Across multiple long-horizon agent benchmarks and models, HAD outperforms on-policy distillation baselines with the same fixed harness. Our analysis shows that HAD enters fewer unproductive loops and recovers from errors more often than the baselines, and suggests that it adaptively keeps learnable feedback in its weights while reading state information from the harness.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.02858 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-05T22:45:55.644Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).