Metraipolis › archive › dc_5cb18593f64f

AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation — arXiv:2609.38142

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.38142
License (as archived)
CC0-1.0
Archived
2026-10-07T16:24:31.293Z
SHA-256 of the archived text
90ff723a25623aa19c6d49ba47931e8590e1f4a15156fc4293e76e40f2b29e01
Stable URL of this page
https://metraipolis.com/documento/dc_5cb18593f64f

How to cite

AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation — arXiv:2609.38142 — arXiv (en), https://arxiv.org/abs/2609.38142 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5cb18593f64f (Metraipolis archive dc_5cb18593f64f, sha256 90ff723a25623aa19c6d49ba47931e8590e1f4a15156fc4293e76e40f2b29e01)
BibTeX
@misc{dc5cb18593f64f,
  title = {AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation — arXiv:2609.38142},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38142},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5cb18593f64f; sha256 90ff723a25623aa19c6d49ba47931e8590e1f4a15156fc4293e76e40f2b29e01},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38142. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A small trainable advisor can steer a frozen language-model executor using natural-language advice. In addition to learning from task rewards, the advisor can use feedback from completed interactions to improve its advice. However, a plausible correction need not change execution, yet learning from such corrections can still affect the advisor's future decisions in other contexts. In a shared-parameter model, we prove that such corrections can limit learning if their targets favor useful advice less strongly than those of other corrections. Keeping them less often than the rest improves the model's eventual performance compared to learning from every correction. Motivated by this, our method, Advisor Self-Distillation (AdviSD), pairs outcome-based reinforcement learning with self-distillation from a feedback-conditioned copy of the advisor selectively. Reflection proposes corrections, and the advisor scores the same recorded executor response with and without its issued advice, using the magnitude of the difference to select decisions for supervision. This approach does not require executor likelihoods or additional executor rollouts. Experiments with Qwen3-8B advisors for Gemini and Claude show that AdviSD outperforms advisor-GRPO by 4.2-6.4 percentage points on BFCL-v3 and by 3.9-5.1 score points on EnvScaler. The trained advisors generalize to out-of-domain tasks and transfer across different executor versions and model families. AdviSD also beats matched-count random selection, supporting the value of its selection rule.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.38142 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.693Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).