Metraipolis › archive › dc_4815803efe4f
ZOPGA-KD: Zeroth Order PGA Guided Data Free Knowledge Distillation — arXiv:2611.00908
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00908
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:53:19.364Z
- SHA-256 of the archived text
bf27dded7bf30a41f7758b651fdd23f15035fd505cd9a8e801c79a99486ec7f5- Stable URL of this page
https://metraipolis.com/documento/dc_4815803efe4f
How to cite
ZOPGA-KD: Zeroth Order PGA Guided Data Free Knowledge Distillation — arXiv:2611.00908 — arXiv (en), https://arxiv.org/abs/2611.00908 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4815803efe4f (Metraipolis archive dc_4815803efe4f, sha256 bf27dded7bf30a41f7758b651fdd23f15035fd505cd9a8e801c79a99486ec7f5)
BibTeX
@misc{dc4815803efe4f,
title = {ZOPGA-KD: Zeroth Order PGA Guided Data Free Knowledge Distillation — arXiv:2611.00908},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00908},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4815803efe4f; sha256 bf27dded7bf30a41f7758b651fdd23f15035fd505cd9a8e801c79a99486ec7f5},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00908. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Knowledge distillation (KD) enables the transfer of learned representations from a large teacher network to a small student network. In foundational KD, the student network uses the original training data of the teacher model to accomplish this task. However, due to privacy concerns, the original training data is often not available. To overcome this reliance on original training data, several data-free knowledge distillation methods have been developed. Some of these methods operate in the white-box setting, and a few methods generate synthetic data by training a generator network. White-box methods use the teacher model's parameters to obtain a synthetic dataset. To avoid relying on teacher parameters, some methods use computationally expensive generative approaches to obtain such a dataset. There also exist certain black-box methods that generate the synthetic data without accessing teacher parameters. These methods are also computationally intensive. We propose a data-free knowledge distillation method that utilizes just the teacher's output logits and does not require access to its architecture, parameters, or gradients. We generate synthetic data via Projected Gradient Ascent (PGA) starting from smooth random noise signals. Required gradients are obtained through zeroth-order gradient estimation rather than backpropagating through the teacher's parameters. The student is then trained on these synthetically generated samples. We evaluate the proposed method on several benchmark datasets and show that the distillation performance is comparable with approaches that have access to the teacher model parameters. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2611.00908 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:16:52.909Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).