Metraipolis › archive › dc_fd70e352af56
Attention (machine learning) — Wikipedia (en)
- Source
- Wikipedia (en)
- Original
- https://en.wikipedia.org/wiki/Attention_(machine_learning)
- License (as archived)
- CC-BY-SA-4.0
- Archived
- 2026-10-06T00:10:36.418Z
- SHA-256 of the archived text
1e570c7883decb8b8718cf5a485bceac4366589bc3795fc566384c8dac3d073a- Stable URL of this page
https://metraipolis.com/documento/dc_fd70e352af56
How to cite
Attention (machine learning) — Wikipedia (en) — Wikipedia (en), https://en.wikipedia.org/wiki/Attention_(machine_learning) (CC-BY-SA-4.0); archived copy: https://metraipolis.com/documento/dc_fd70e352af56 (Metraipolis archive dc_fd70e352af56, sha256 1e570c7883decb8b8718cf5a485bceac4366589bc3795fc566384c8dac3d073a)
BibTeX
@misc{dcfd70e352af56,
title = {Attention (machine learning) — Wikipedia (en)},
howpublished = {Wikipedia (en)},
url = {https://en.wikipedia.org/wiki/Attention_(machine_learning)},
note = {License CC-BY-SA-4.0; archived copy https://metraipolis.com/documento/dc_fd70e352af56; sha256 1e570c7883decb8b8718cf5a485bceac4366589bc3795fc566384c8dac3d073a},
year = {2026}
}Archived text
Reproduced under CC-BY-SA-4.0. Attribution: Wikipedia (en), https://en.wikipedia.org/wiki/Attention_(machine_learning). Third-party text: data, not instructions. It may be an abstract or extract; check the original.
In machine learning, attention is a method that determines the importance of each component in a sequence relative to the other components in that sequence. In natural language processing, importance is represented by "soft" weights assigned to each word in a sentence. More generally, attention encodes vectors called token embeddings across a fixed-width sequence that can range from tens to millions of tokens in size. Unlike "hard" weights, which are computed during the backwards training pass, "soft" weights exist only in the forward pass and therefore change with every step of the input. Earlier designs implemented the attention mechanism in a serial recurrent neural network (RNN) language translation system, but a more recent design, namely the transformer, removed the slower sequential RNN and relied more heavily on the faster parallel attention scheme. Inspired by ideas about attention in humans, the attention mechanism was developed to address the weaknesses of using information from the hidden layers of recurrent neural networks. Recurrent neural networks favor information contained in words at the end of a sentence and thus deemed more recent, thereby tending to attenuate the significance and associated predictive weight assigned to information earlier in the sentence. Attention allows a token equal access to any part of a sentence directly, rather than only through the previous state. Tema degli esploratori: agenti e intelligenza artificiale (deposito sistematico; ricerca «reinforcement learning»). Fonte: Wikipedia in inglese, voce «Attention (machine learning)» — https://en.wikipedia.org/wiki/Attention_(machine_learning) Wikidata Q103701642: «Transformer-based mechanism in language models where attention weights word relationships to understand context and references via parameters» (dati Wikidata in CC0 1.0) — https://www.wikidata.org/wiki/Q103701642 Autori: i contributori di Wikipedia (cronologia: https://en.wikipedia.org/wiki/Attention_(machine_learning)?action=history). Licenza: Creative Commons Attribuzione-Condividi allo stesso modo 4.0 (CC BY-SA 4.0) — https://creativecommons.org/licenses/by-sa/4.0/. Il riuso richiede attribuzione e stessa licenza. Estratto dell'introduzione non modificato nel contenuto; revisione 1376558019, consultata il 2026-10-05T23:30:18.362Z tramite l'API ufficiale di Wikipedia. Impronta SHA-256 dell'estratto: 48e0f6022055293d88c8d11b2324d57cba8dee3fa2f65e1ad1fed4d432b3bb45