Metraipolis › archive › dc_8b84fd93191f

Small Language Models: Survey, Measurements, and Insights (2024) · OpenAlex W4403786771

Source
OpenAlex
Original
https://doi.org/10.48550/arxiv.2409.15790
License (as archived)
CC-BY-4.0
Archived
2026-10-08T03:23:40.674Z
SHA-256 of the archived text
f4fb343009a6aa76e554dc6e152a9e7e630b3e77dd15b0a8ec09484c70dc13c5
Stable URL of this page
https://metraipolis.com/documento/dc_8b84fd93191f

How to cite

Small Language Models: Survey, Measurements, and Insights (2024) · OpenAlex W4403786771 — OpenAlex, https://doi.org/10.48550/arxiv.2409.15790 (CC-BY-4.0); archived copy: https://metraipolis.com/documento/dc_8b84fd93191f (Metraipolis archive dc_8b84fd93191f, sha256 f4fb343009a6aa76e554dc6e152a9e7e630b3e77dd15b0a8ec09484c70dc13c5)
BibTeX
@misc{dc8b84fd93191f,
  title = {Small Language Models: Survey, Measurements, and Insights (2024) · OpenAlex W4403786771},
  howpublished = {OpenAlex},
  url = {https://doi.org/10.48550/arxiv.2409.15790},
  note = {License CC-BY-4.0; archived copy https://metraipolis.com/documento/dc_8b84fd93191f; sha256 f4fb343009a6aa76e554dc6e152a9e7e630b3e77dd15b0a8ec09484c70dc13c5},
  year = {2026}
}

Archived text

Reproduced under CC-BY-4.0. Attribution: OpenAlex, https://doi.org/10.48550/arxiv.2409.15790. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Opera scientifica in accesso aperto: scheda dai metadati OpenAlex (preprint).
Titolo: Small Language Models: Survey, Measurements, and Insights
Autori: Zhichun Lu, Li, Xiang, Cai, Dongqi, Runduan Yi, Fangming Liu, Xiwen Zhang, Nicholas D. Lane, Mengwei Xu
Anno: 2024 · Sede: arXiv (Cornell University) · Lingua: en
DOI: https://doi.org/10.48550/arxiv.2409.15790
Accesso aperto: green · testo: http://arxiv.org/abs/2409.15790 · licenza del testo: cc-by
Citazioni ricevute (OpenAlex): 19
Temi (OpenAlex): Topic Modeling; Natural Language Processing Techniques; Advanced Neural Network Applications

Abstract (testo degli autori, licenza cc-by):
Small language models (SLMs), despite their widespread adoption in modern smart devices, have received significantly less academic attention compared to their large language model (LLM) counterparts, which are predominantly deployed in data centers and cloud environments. While researchers continue to improve the capabilities of LLMs in the pursuit of artificial general intelligence, SLM research aims to make machine intelligence more accessible, affordable, and efficient for everyday tasks. Focusing on transformer-based, decoder-only language models with 100M-5B parameters, we survey 70 state-of-the-art open-source SLMs, analyzing their technical innovations across three axes: architectures, training datasets, and training algorithms. In addition, we evaluate their capabilities in various domains, including commonsense reasoning, mathematics, in-context learning, and long context. To gain further insight into their on-device runtime costs, we benchmark their inference latency and memory footprints. Through in-depth analysis of our benchmarking data, we offer valuable insights to advance research in this field.

Fonte: https://openalex.org/W4403786771 — metadati OpenAlex (CC0 1.0), consultati il 2026-10-07 via api.openalex.org. Interrogazione: «small language models».