Metraipolis › archive › dc_1817839ef4fb
Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation — arXiv:2609.13199
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13199
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T00:10:15.650Z
- SHA-256 of the archived text
5ab2856eb9325cfdcdb06215a6c2860975542e62f161fdaefe31aa581b3e630f- Stable URL of this page
https://metraipolis.com/documento/dc_1817839ef4fb
How to cite
Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation — arXiv:2609.13199 — arXiv (en), https://arxiv.org/abs/2609.13199 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1817839ef4fb (Metraipolis archive dc_1817839ef4fb, sha256 5ab2856eb9325cfdcdb06215a6c2860975542e62f161fdaefe31aa581b3e630f)
BibTeX
@misc{dc1817839ef4fb,
title = {Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation — arXiv:2609.13199},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13199},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1817839ef4fb; sha256 5ab2856eb9325cfdcdb06215a6c2860975542e62f161fdaefe31aa581b3e630f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13199. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher models. However, a substantial size gap between teacher and student models often impedes effective knowledge transfer. Most existing approaches adopt a static, one-way teacher-to-student distillation paradigm, which overlooks the dynamic nature of student learning and fails to provide targeted guidance on hard samples. In this paper, we propose adaptive reciprocal knowledge distillation (AR-KD), a novel method that improves knowledge transfer by simplifying the teacher's output distribution. Specifically, AR-KD performs reciprocal adaptation on the teacher by matching its class correlation matrix to the student's relational representation, which reshapes the teacher's prediction structure to better suit the student's capacity. This relational alignment mitigates the collapse of inter-class dark knowledge caused by overconfident teachers, enabling the student to learn from richer and more compatible supervisory signals. We evaluate AR-KD on CIFAR-100 and ImageNet-1k classification datasets, where it outperforms state-of-the-art knowledge distillation baselines. Specifically, AR-KD improves student performance across homogeneous and heterogeneous setups: up to 7.13% accuracy gain for students, 1.42% to 4.15% higher than vanilla KD on average, and further improvements when integrated with other advanced methods. Our code is available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-15 in cs.LG). Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.13199 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-15T23:54:14.602Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-15?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).