Metraipolis › archive › dc_89c707bbfad4

Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation — arXiv:2609.39120

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39120
License (as archived)
CC0-1.0
Archived
2026-10-07T14:45:51.073Z
SHA-256 of the archived text
d8de9dea3135f6c50ed992f0038c9504c181498b30f980b0c717ff5e91c90bc1
Stable URL of this page
https://metraipolis.com/documento/dc_89c707bbfad4

How to cite

Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation — arXiv:2609.39120 — arXiv (en), https://arxiv.org/abs/2609.39120 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_89c707bbfad4 (Metraipolis archive dc_89c707bbfad4, sha256 d8de9dea3135f6c50ed992f0038c9504c181498b30f980b0c717ff5e91c90bc1)
BibTeX
@misc{dc89c707bbfad4,
  title = {Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation — arXiv:2609.39120},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39120},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_89c707bbfad4; sha256 d8de9dea3135f6c50ed992f0038c9504c181498b30f980b0c717ff5e91c90bc1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39120. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy distillation (OPD) improves reasoning by providing token-level supervision from a teacher on a student's own trajectories. Existing methods primarily focus on enhancing this teacher-side guidance (e.g., by enriching teacher inputs and refining teacher feedback), yet we find that limited student perception is another critical bottleneck in multimodal OPD. By providing oracle visual facts, the performance of OPD-trained students can still be substantially improved for both weak and strong teachers. To address this bottleneck, we propose S-OPD, a simple multimodal on-policy distillation framework that explicitly strengthens student perceptual learning through two objectives. Specifically, Teacher-calibrated Policy Contrast separates student policies under original and masked images with teacher-based token-level gating, strengthening the student's reliance on visual evidence during reasoning. Policy Agreement aligns student policies under original and noise-perturbed images, further improving perceptual robustness to visual noise. Notably, our method can be seamlessly plugged into existing OPD frameworks, requiring no additional data annotations, model parameters or inference operations. Extensive experiments on eight benchmarks across student scales and distillation paradigms demonstrate consistent performance improvements, with gains of up to 4.25 points on LogicVista. When combined with existing teacher-side supervision methods, our method can yield further gains. Code is available at this https URL .

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-10-01 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.39120 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T14:19:00.676Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).