Metraipolis › archive › dc_22259c2f01dc
Enhanced Knowledge Distillation for Detection Transformer via Teacher Prediction Refinement — arXiv:2609.19964
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.19964
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T05:05:38.281Z
- SHA-256 of the archived text
d820a8634fbf6cf03afece89ae54a65cb7932b540f2c34e1235297cbb95fba8f- Stable URL of this page
https://metraipolis.com/documento/dc_22259c2f01dc
How to cite
Enhanced Knowledge Distillation for Detection Transformer via Teacher Prediction Refinement — arXiv:2609.19964 — arXiv (en), https://arxiv.org/abs/2609.19964 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_22259c2f01dc (Metraipolis archive dc_22259c2f01dc, sha256 d820a8634fbf6cf03afece89ae54a65cb7932b540f2c34e1235297cbb95fba8f)
BibTeX
@misc{dc22259c2f01dc,
title = {Enhanced Knowledge Distillation for Detection Transformer via Teacher Prediction Refinement — arXiv:2609.19964},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.19964},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_22259c2f01dc; sha256 d820a8634fbf6cf03afece89ae54a65cb7932b540f2c34e1235297cbb95fba8f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19964. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Detection Transformers (DETRs) achieve strong performance in object detection but remain challenging to deploy on edge devices due to their high computational cost. Existing DETR distillation methods mainly focus on aligning distillation points, while largely overlooking the quality of the teacher's supervision itself. We observe that due to stage-wise non-monotonic prediction behavior in DETRs, well-localized or correctly classified predictions from earlier stages may degrade in later ones, and some negative predictions become increasingly overconfident. As a result, relying solely on the current stage's predictions yields inaccurate and inconsistent supervision. To address this issue, we propose Teacher Prediction Refinement Distillation (TPRD), a plug-and-play module that refines teacher predictions before distillation by exploiting stage-wise prediction information. TPRD improves supervision quality through Positive Prediction Correction (PPC), which corrects degraded positive predictions by restoring more accurate ones from earlier stages, ensuring reliable localization and classification signals, and Negative Prediction Suppression (NPS) suppresses the influence of overconfident negatives, preventing them from providing misleading supervision to the student. To preserve informative dark knowledge, we further introduce Maximum Dark Knowledge Preservation (MDKP), which selectively refines target-class logits while retaining non-target relations. Extensive experiments on MS COCO and PASCAL VOC demonstrate the effectiveness and robustness of the proposed method. Our code is available at this https URL . Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-18 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.19964 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T04:26:09.565Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).