Metraipolis › archive › dc_434cc7ad2fc4

Disentangling Language Modeling and Boundaries — arXiv:2608.03599

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.03599
License (as archived)
CC0-1.0
Archived
2026-10-31T12:31:07.898Z
SHA-256 of the archived text
82dfbfd997a6cc0b6fa51fe7e803e86a48a3f53ca770b7bd75952b529767d9d4
Stable URL of this page
https://metraipolis.com/documento/dc_434cc7ad2fc4

How to cite

Disentangling Language Modeling and Boundaries — arXiv:2608.03599 — arXiv (en), https://arxiv.org/abs/2608.03599 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_434cc7ad2fc4 (Metraipolis archive dc_434cc7ad2fc4, sha256 82dfbfd997a6cc0b6fa51fe7e803e86a48a3f53ca770b7bd75952b529767d9d4)
BibTeX
@misc{dc434cc7ad2fc4,
  title = {Disentangling Language Modeling and Boundaries — arXiv:2608.03599},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.03599},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_434cc7ad2fc4; sha256 82dfbfd997a6cc0b6fa51fe7e803e86a48a3f53ca770b7bd75952b529767d9d4},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.03599. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Byte-level language models are usually argued for on the grounds of robustness, multilingual fairness, and character-level skills. We point to a different, structural advantage: because they read and write bytes, any two of them share an output space, so knowledge transfer between them is exact and independent of how either was originally tokenized. We hypothesize that the two distributions a byte-level model produces, one over the next byte, one over where its patch boundaries fall, can be disentangled and changed almost independently. A model could absorb a teacher's capability while keeping its own boundaries, or change how it places those boundaries while keeping its capabilities. We lay out the two experiments that would settle the hypothesis, alongside preliminary measurements of the properties they rest on. We argue that the community should move toward a byte-level interface as a shared standard: if the hypothesis holds, then once byte-level models are the norm, transferring capabilities and reshaping boundaries between them become cheap and routine, free of the per-model tokenizer that blocks them today.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.03599 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:17:37.771Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).