Metraipolis › archive › dc_a692e32e89c5
Self-Supervised Visual On-Policy Distillation — arXiv:2608.14144
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.14144
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T02:21:24.573Z
- SHA-256 of the archived text
0b0dd668bb30f5003fa180b70172b3c8ad561e2b18088c29a0340a68b47ea740- Stable URL of this page
https://metraipolis.com/documento/dc_a692e32e89c5
How to cite
Self-Supervised Visual On-Policy Distillation — arXiv:2608.14144 — arXiv (en), https://arxiv.org/abs/2608.14144 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a692e32e89c5 (Metraipolis archive dc_a692e32e89c5, sha256 0b0dd668bb30f5003fa180b70172b3c8ad561e2b18088c29a0340a68b47ea740)
BibTeX
@misc{dca692e32e89c5,
title = {Self-Supervised Visual On-Policy Distillation — arXiv:2608.14144},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.14144},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a692e32e89c5; sha256 0b0dd668bb30f5003fa180b70172b3c8ad561e2b18088c29a0340a68b47ea740},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.14144. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Visual on-policy distillation relies heavily on an informative teacher-student asymmetry, through either a larger, stronger teacher or privileged supervision, such as reference answers or ground-truth regions of interest. This raises a fundamental question: where can informative asymmetry come from when nothing privileged is available? We answer this by inverting where the asymmetry comes from. Rather than adding privileged information to the teacher, we subtract information from the student. This asymmetry creates the same effective learning signal for free as a teacher with access to information unavailable to the student, without ground-truth annotations, rewards, or a separate stronger teacher model. Building on this principle, we introduce Self-Supervised Visual On-Policy Distillation (S$^2$VOPD), a simple yet effective method that constructs on-policy learning signals from asymmetric augmented views. S$^2$VOPD distills the teacher's distribution conditioned on the original image on-policy into the student distribution conditioned on a strongly augmented view of the same image. We systematically explore a broad design space of visual augmentations and uncover that (1) asymmetry matters: all four augmentation families improve performance, while symmetric self-distillation degrades it; (2) strength matters: performance peaks at a moderate strength; and (3) the gap must remain task-consistent: augmentations that completely remove the question-relevant evidence can induce large but uninformative discrepancies. Across six fine-grained perception benchmarks, S$^2$VOPD improves Qwen3.5-4B from 70.7% to 77.4%, above all open-source models compared, up to Qwen3-VL at 235B, and surpasses GPT-5.4. While holding training data the same, it recovers 96% of the improvement achieved by methods with privileged information. Website is at this https URL Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.AI). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.14144 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T02:02:02.920Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).