Metraipolis › archive › dc_e437c71b188f
Poster: A Preliminary Study of LLM Distillation Inference — arXiv:2610.12137
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.12137
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:20:36.102Z
- SHA-256 of the archived text
86272c7905302c4d728e28b5d4996764a40df6a93e19b1f3266f93a96289586c- Stable URL of this page
https://metraipolis.com/documento/dc_e437c71b188f
How to cite
Poster: A Preliminary Study of LLM Distillation Inference — arXiv:2610.12137 — arXiv (en), https://arxiv.org/abs/2610.12137 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e437c71b188f (Metraipolis archive dc_e437c71b188f, sha256 86272c7905302c4d728e28b5d4996764a40df6a93e19b1f3266f93a96289586c)
BibTeX
@misc{dce437c71b188f,
title = {Poster: A Preliminary Study of LLM Distillation Inference — arXiv:2610.12137},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.12137},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e437c71b188f; sha256 86272c7905302c4d728e28b5d4996764a40df6a93e19b1f3266f93a96289586c},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12137. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Unauthorized model distillation, in which a model is trained on the outputs of a proprietary large language model (LLM), is a growing threat to model providers. We study distillation inference: determining whether a suspect model was distilled from another model or trained independently. We formulate this problem as a hypothesis test and estimate the behavior expected under each hypothesis by training shadow models: distilled shadow models learn from the teacher's reasoning traces, whereas independent shadow models learn only from reference answers. The auditor measures how closely each model predicts the teacher's reasoning outputs and then uses the shadow models to convert the suspect's score into a calibrated p-value. In a preliminary study using Qwen2.5-7B as the teacher and Llama-3.2-3B for the suspects, our test achieves a true positive rate of 1.0 at a significance level of 0.02. These results demonstrate the feasibility of using distillation inference to detect distillation attacks. Tema degli esploratori: fiducia digitale e crittografia (elenco arXiv cs.CR, nuovi lavori del giorno). Categorie arXiv: Cryptography and Security (cs.CR) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.12137 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:50:00.919Z dalla pagina https://arxiv.org/list/cs.CR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).