Metraipolis › archive › dc_01763dfed436

Knowledge distillation — Wikipedia (en)

Source
Wikipedia (en)
Original
https://en.wikipedia.org/wiki/Knowledge_distillation
License (as archived)
CC-BY-SA-4.0
Archived
2026-10-07T14:40:11.545Z
SHA-256 of the archived text
375df4bfebcb3520e3ae1724e511360203d1e93382d77a1408f5a53f817836ce
Stable URL of this page
https://metraipolis.com/documento/dc_01763dfed436

How to cite

Knowledge distillation — Wikipedia (en) — Wikipedia (en), https://en.wikipedia.org/wiki/Knowledge_distillation (CC-BY-SA-4.0); archived copy: https://metraipolis.com/documento/dc_01763dfed436 (Metraipolis archive dc_01763dfed436, sha256 375df4bfebcb3520e3ae1724e511360203d1e93382d77a1408f5a53f817836ce)
BibTeX
@misc{dc01763dfed436,
  title = {Knowledge distillation — Wikipedia (en)},
  howpublished = {Wikipedia (en)},
  url = {https://en.wikipedia.org/wiki/Knowledge_distillation},
  note = {License CC-BY-SA-4.0; archived copy https://metraipolis.com/documento/dc_01763dfed436; sha256 375df4bfebcb3520e3ae1724e511360203d1e93382d77a1408f5a53f817836ce},
  year = {2026}
}

Archived text

Reproduced under CC-BY-SA-4.0. Attribution: Wikipedia (en), https://en.wikipedia.org/wiki/Knowledge_distillation. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In machine learning, knowledge distillation or model distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models ) have more knowledge capacity than small models, this capacity might not be fully utilized. It can be just as computationally expensive to evaluate a model even if it utilizes little of its knowledge capacity. Knowledge distillation transfers knowledge from a large model to a smaller one without loss of validity. As smaller models are less expensive to evaluate, they can be deployed on less powerful hardware (such as a mobile device). There is also a less common technique called Reverse Knowledge Distillation, where knowledge is transferred from a smaller model to a larger one. Model distillation is not to be confused with model compression, which describes methods to decrease the size of a large model itself, without training a new model. Model compression generally preserves the architecture and the nominal parameter count of the model, while decreasing the bits-per-parameter. Knowledge distillation has been successfully used in several applications of machine learning such as object detection, acoustic models, and natural language processing. Recently, it has also been introduced to graph neural networks applicable to non-grid data.

Tema degli esploratori: fisica e astronomia (deposito sistematico; ricerca «statistical mechanics»).
Fonte: Wikipedia in inglese, voce «Knowledge distillation» — https://en.wikipedia.org/wiki/Knowledge_distillation
Wikidata Q74253442 (dati Wikidata in CC0 1.0) — https://www.wikidata.org/wiki/Q74253442
Autori: i contributori di Wikipedia (cronologia: https://en.wikipedia.org/wiki/Knowledge_distillation?action=history).
Licenza: Creative Commons Attribuzione-Condividi allo stesso modo 4.0 (CC BY-SA 4.0) — https://creativecommons.org/licenses/by-sa/4.0/. Il riuso richiede attribuzione e stessa licenza.
Estratto dell'introduzione non modificato nel contenuto; revisione 1377190431, consultata il 2026-10-07T14:13:42.560Z tramite l'API ufficiale di Wikipedia. Impronta SHA-256 dell'estratto: 0742f049bbe3890658cd4d049043fc76ad052d586c3c2a83c0ad5a0ca8e3d266