Metraipolis › archive › dc_bf368dc80d26

Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation — arXiv:2611.08179

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.08179
License (as archived)
CC0-1.0
Archived
2026-11-06T14:42:51.201Z
SHA-256 of the archived text
3fe51c99918c3b546e70109be0bd53ab26a047b09b36de6102a8359858cbaecd
Stable URL of this page
https://metraipolis.com/documento/dc_bf368dc80d26

How to cite

Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation — arXiv:2611.08179 — arXiv (en), https://arxiv.org/abs/2611.08179 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bf368dc80d26 (Metraipolis archive dc_bf368dc80d26, sha256 3fe51c99918c3b546e70109be0bd53ab26a047b09b36de6102a8359858cbaecd)
BibTeX
@misc{dcbf368dc80d26,
  title = {Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation — arXiv:2611.08179},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.08179},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bf368dc80d26; sha256 3fe51c99918c3b546e70109be0bd53ab26a047b09b36de6102a8359858cbaecd},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08179. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-Policy Self-Distillation (OPSD) improves reasoning by training a student on its own reasoning, using the same model conditioned on reference solutions as a teacher. Applying OPSD to multilingual reasoning requires the teacher to provide useful feedback across languages, even when the student makes mistakes in intermediate steps. We observe a phenomenon we call Teacher Drift: as these mistakes accumulate, the teacher may follow the student's incorrect steps instead of correcting them. Our analysis shows that this shift tends to occur earlier in lower-resource languages. To account for these differences across languages, we propose Teachable On-Policy Self-Distillation (TOPSD), which determines where and whether teacher supervision should be applied to each reasoning trajectory. TOPSD uses teacher-student likelihood to estimate where drift begins and select a prefix for supervision. It then verifies whether the teacher can recover the correct answer from that prefix, applying distillation only to prefixes that pass verification. Experiments on multilingual mathematical reasoning across ten languages and three model scales show that TOPSD consistently outperforms conventional OPSD, particularly in low- and medium-resource languages.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.08179 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:18:42.698Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).