Metraipolis › archive › dc_f78bfacedf16
ZOPGA-KD: Zeroth Order PGA Guided Data Free Knowledge Distillation — arXiv:2611.00908
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00908
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:40:50.558Z
- SHA-256 of the archived text
d4e421629a61dcba1b9f2dbe18588d906f1b589dbf3992386ccf96b5efd4da3f- Stable URL of this page
https://metraipolis.com/documento/dc_f78bfacedf16
How to cite
ZOPGA-KD: Zeroth Order PGA Guided Data Free Knowledge Distillation — arXiv:2611.00908 — arXiv (en), https://arxiv.org/abs/2611.00908 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f78bfacedf16 (Metraipolis archive dc_f78bfacedf16, sha256 d4e421629a61dcba1b9f2dbe18588d906f1b589dbf3992386ccf96b5efd4da3f)
BibTeX
@misc{dcf78bfacedf16,
title = {ZOPGA-KD: Zeroth Order PGA Guided Data Free Knowledge Distillation — arXiv:2611.00908},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00908},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f78bfacedf16; sha256 d4e421629a61dcba1b9f2dbe18588d906f1b589dbf3992386ccf96b5efd4da3f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00908. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Knowledge distillation (KD) enables the transfer of learned representations from a large teacher network to a small student network. In foundational KD, the student network uses the original training data of the teacher model to accomplish this task. However, due to privacy concerns, the original training data is often not available. To overcome this reliance on original training data, several data-free knowledge distillation methods have been developed. Some of these methods operate in the white-box setting, and a few methods generate synthetic data by training a generator network. White-box methods use the teacher model's parameters to obtain a synthetic dataset. To avoid relying on teacher parameters, some methods use computationally expensive generative approaches to obtain such a dataset. There also exist certain black-box methods that generate the synthetic data without accessing teacher parameters. These methods are also computationally intensive. We propose a data-free knowledge distillation method that utilizes just the teacher's output logits and does not require access to its architecture, parameters, or gradients. We generate synthetic data via Projected Gradient Ascent (PGA) starting from smooth random noise signals. Required gradients are obtained through zeroth-order gradient estimation rather than backpropagating through the teacher's parameters. The student is then trained on these synthetically generated samples. We evaluate the proposed method on several benchmark datasets and show that the distillation performance is comparable with approaches that have access to the teacher model parameters. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2611.00908 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:06:37.459Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).