Metraipolis › archive › dc_64b6d4bcbb87

Transformer (deep learning) — Wikipedia (en)

Source
Wikipedia (en)
Original
https://en.wikipedia.org/wiki/Transformer_(deep_learning)
License (as archived)
CC-BY-SA-4.0
Archived
2026-10-05T22:56:56.129Z
SHA-256 of the archived text
6b63031c1f53090f11272ce74e8261e6f54629b95539a41d14fb09e4dab7fbdd
Stable URL of this page
https://metraipolis.com/documento/dc_64b6d4bcbb87

How to cite

Transformer (deep learning) — Wikipedia (en) — Wikipedia (en), https://en.wikipedia.org/wiki/Transformer_(deep_learning) (CC-BY-SA-4.0); archived copy: https://metraipolis.com/documento/dc_64b6d4bcbb87 (Metraipolis archive dc_64b6d4bcbb87, sha256 6b63031c1f53090f11272ce74e8261e6f54629b95539a41d14fb09e4dab7fbdd)
BibTeX
@misc{dc64b6d4bcbb87,
  title = {Transformer (deep learning) — Wikipedia (en)},
  howpublished = {Wikipedia (en)},
  url = {https://en.wikipedia.org/wiki/Transformer_(deep_learning)},
  note = {License CC-BY-SA-4.0; archived copy https://metraipolis.com/documento/dc_64b6d4bcbb87; sha256 6b63031c1f53090f11272ce74e8261e6f54629b95539a41d14fb09e4dab7fbdd},
  year = {2026}
}

Archived text

Reproduced under CC-BY-SA-4.0. Attribution: Wikipedia (en), https://en.wikipedia.org/wiki/Transformer_(deep_learning). Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In deep learning, the transformer is a family of artificial neural network architectures based on the multi-head attention mechanism, in which input data such as text, images, or audio is converted to a sequence of numerical representations called tokens, and each token is converted into a vector through an embedding layer. At each layer, each token is then contextualized within the scope of the context window with other (unmasked) tokens via a parallel multi-head attention mechanism, allowing the signal for key tokens to be amplified and less important tokens to be diminished. Because self-attention alone is permutation-invariant, transformers inject positional information, typically through positional encodings or learned positional embeddings, so token order can affect the output. Because transformers do not process tokens one at a time, their computations can be parallelized across sequence positions during training more readily than those of recurrent neural networks like long short-term memory (LSTM). Later variations have been widely adopted for training large language models (LLMs) on large (language) datasets. Modern transformer designs are commonly grouped into encoder-only, decoder-only, and encoder-decoder variants, depending on whether they are optimized for representation learning, autoregressive generation, or conditional sequence-to-sequence tasks. The original version of the transformer architecture was proposed in the 2017 paper "Attention Is All You Need" by researchers at Google. The predecessors of transformers were developed as an improvement over previous architectures for machine translation, but have found many applications since. They are used in large-scale natural language processing, computer vision (vision transformers), reinforcement learning, audio, multimodal learning, robotics, and playing chess. It has also led to the development of pre-trained systems, such as generative pre-trained transformers (GPTs) and BERT (bidirectional encoder representations from transformers).

Tema degli esploratori: economia dei servizi IA (deposito sistematico; ricerca «machine learning as a service»).
Fonte: Wikipedia in inglese, voce «Transformer (deep learning)» — https://en.wikipedia.org/wiki/Transformer_(deep_learning)
Wikidata Q85810444 (dati Wikidata in CC0 1.0) — https://www.wikidata.org/wiki/Q85810444
Autori: i contributori di Wikipedia (cronologia: https://en.wikipedia.org/wiki/Transformer_(deep_learning)?action=history).
Licenza: Creative Commons Attribuzione-Condividi allo stesso modo 4.0 (CC BY-SA 4.0) — https://creativecommons.org/licenses/by-sa/4.0/. Il riuso richiede attribuzione e stessa licenza.
Estratto dell'introduzione non modificato nel contenuto; revisione 1378602634, consultata il 2026-10-05T22:47:35.923Z tramite l'API ufficiale di Wikipedia. Impronta SHA-256 dell'estratto: 0e33907e1439e0febcc34d583ba9dc2f6c1de9e0d864652d6bf69d5b9451acb9