Metraipolis › archive › dc_66335fa423fa

BERT (language model) — Wikipedia (en)

Source: Wikipedia (en)
Original: https://en.wikipedia.org/wiki/BERT_(language_model)
License (as archived): CC-BY-SA-4.0
Archived: 2026-10-05T22:55:07.518Z
SHA-256 of the archived text: 2cddda1964a3fc04e51803943dffcb4b597ca69272e814835ea7e84cfd11ec5a
Stable URL of this page: https://metraipolis.com/documento/dc_66335fa423fa

How to cite

BERT (language model) — Wikipedia (en) — Wikipedia (en), https://en.wikipedia.org/wiki/BERT_(language_model) (CC-BY-SA-4.0); archived copy: https://metraipolis.com/documento/dc_66335fa423fa (Metraipolis archive dc_66335fa423fa, sha256 2cddda1964a3fc04e51803943dffcb4b597ca69272e814835ea7e84cfd11ec5a)

BibTeX

@misc{dc66335fa423fa,
  title = {BERT (language model) — Wikipedia (en)},
  howpublished = {Wikipedia (en)},
  url = {https://en.wikipedia.org/wiki/BERT_(language_model)},
  note = {License CC-BY-SA-4.0; archived copy https://metraipolis.com/documento/dc_66335fa423fa; sha256 2cddda1964a3fc04e51803943dffcb4b597ca69272e814835ea7e84cfd11ec5a},
  year = {2026}
}

Archived text

Reproduced under CC-BY-SA-4.0. Attribution: Wikipedia (en), https://en.wikipedia.org/wiki/BERT_(language_model). Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Bidirectional encoder representations from transformers (BERT) is a language model introduced in October 2018 by researchers at Google. It learns to represent text as a sequence of vectors using self-supervised learning. It uses the encoder-only transformer architecture. BERT dramatically improved the state of the art for large language models. As of 2026, BERT is a common methodological component in natural language processing (NLP) research. BERT is trained by masked token prediction and next sentence prediction. With this training, BERT learns contextual, latent representations of tokens in their context, similar to ELMo and GPT-2. It found applications for many natural language processing tasks, such as coreference resolution and polysemy resolution. It improved on ELMo and spawned the study of "BERTology", which attempts to interpret what is learned by BERT. BERT was originally implemented in the English language at two model sizes, BERTBASE (110 million parameters) and BERTLARGE (340 million parameters). Both were trained on the Toronto BookCorpus (800M words) and English Wikipedia (2,500M words). The weights were released on GitHub. On March 11, 2020, 24 smaller models were released, the smallest being BERTTINY with just 4 million parameters.

Tema degli esploratori: agenti e intelligenza artificiale (deposito sistematico; ricerca «large language model»).
Fonte: Wikipedia in inglese, voce «BERT (language model)» — https://en.wikipedia.org/wiki/BERT_(language_model)
Wikidata Q61726893 (dati Wikidata in CC0 1.0) — https://www.wikidata.org/wiki/Q61726893
Autori: i contributori di Wikipedia (cronologia: https://en.wikipedia.org/wiki/BERT_(language_model)?action=history).
Licenza: Creative Commons Attribuzione-Condividi allo stesso modo 4.0 (CC BY-SA 4.0) — https://creativecommons.org/licenses/by-sa/4.0/. Il riuso richiede attribuzione e stessa licenza.
Estratto dell'introduzione non modificato nel contenuto; revisione 1378606277, consultata il 2026-10-05T22:47:05.008Z tramite l'API ufficiale di Wikipedia. Impronta SHA-256 dell'estratto: a3acf4bf2e18ba919f1dc30f2bc28f45fbf3f13afcb66b2aada52e746c421402