Metraipolis › archive › dc_d5ca882cab5b
The Life of a Token: from Words to Bits on the Wire — arXiv:2609.19924
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.19924
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T03:48:55.118Z
- SHA-256 of the archived text
40cfa31a7d0464824f58a3a9add39b107efb519fea08acc1db9942a128bb274e- Stable URL of this page
https://metraipolis.com/documento/dc_d5ca882cab5b
How to cite
The Life of a Token: from Words to Bits on the Wire — arXiv:2609.19924 — arXiv (en), https://arxiv.org/abs/2609.19924 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d5ca882cab5b (Metraipolis archive dc_d5ca882cab5b, sha256 40cfa31a7d0464824f58a3a9add39b107efb519fea08acc1db9942a128bb274e)
BibTeX
@misc{dcd5ca882cab5b,
title = {The Life of a Token: from Words to Bits on the Wire — arXiv:2609.19924},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.19924},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d5ca882cab5b; sha256 40cfa31a7d0464824f58a3a9add39b107efb519fea08acc1db9942a128bb274e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19924. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Large Language Models (LLMs) transform vast collections of unstructured text into semantic patterns used for language generation and reasoning tasks. Behind their ease of use lies a complex process: words become tokens, tokens become vectors, and vectors ultimately give rise to streams of bits that flow through High-Performance Computing (HPC) systems. As modern LLMs grow to billions or trillions of parameters, this path increasingly unfolds across thousands of interconnected accelerators, making the underlying communication fabric a critical and often opaque component of model training. This tutorial aims to walk the reader through the journey from words to network traffic, shedding light on how language is translated into communication flows within HPC training systems. Using concrete examples from Dante's Divine Comedy, we illustrate how model architecture, tokenization, embeddings, and parallelization strategies shape the volume, structure, and timing of data exchanged across the network. We combine architectural analysis with analytical traffic models and numerical examples to characterize the communication requirements of LLM training. We try to demystify how words travel across the network and provide practical insights into the network requirements needed to support the journey from text to trained model. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-18 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2609.19924 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:24:19.918Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).