Metraipolis › archive › dc_a2cdc604aa6b

X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding — arXiv:2609.09166

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.09166
License (as archived): CC0-1.0
Archived: 2026-10-16T07:50:50.454Z
SHA-256 of the archived text: f0653902ada004937c9f598c8b0a6325d046e6f4b85a10d8130fe3e7cb49cba1
Stable URL of this page: https://metraipolis.com/documento/dc_a2cdc604aa6b

How to cite

X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding — arXiv:2609.09166 — arXiv (en), https://arxiv.org/abs/2609.09166 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a2cdc604aa6b (Metraipolis archive dc_a2cdc604aa6b, sha256 f0653902ada004937c9f598c8b0a6325d046e6f4b85a10d8130fe3e7cb49cba1)

BibTeX

@misc{dca2cdc604aa6b,
  title = {X-CoSD: Communication-Efficient Cross-Vocabulary Collaborative Speculative Decoding — arXiv:2609.09166},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.09166},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a2cdc604aa6b; sha256 f0653902ada004937c9f598c8b0a6325d046e6f4b85a10d8130fe3e7cb49cba1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.09166. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

This paper investigates collaborative speculative decoding (CoSD), a distributed large language model (LLM) inference framework in which an on-device small language model (SLM) drafts candidate tokens and a server LLM verifies them. Existing CoSD methods assume a shared vocabulary between the SLM and the LLM and incur substantial communication load because residual resampling requires token distribution exchange between the user device and the edge server. To address these limitations, we propose cross-vocabulary CoSD (X-CoSD), a lossless and communication-efficient CoSD framework for heterogeneous SLM-LLM vocabularies. X-CoSD is built on hybrid resampling (HR), which splits residual resampling across the common-vocabulary region on the device and the LLM-only region on the server, so that distribution transmission is required only for the common-vocabulary region. We further propose X-CoSD-E, an enhanced variant based on server resampling with device verification (SR-DV), in which the server sends only replacement candidates sampled from the server LLM and their corresponding probabilities for local verification at the device. We prove that both X-CoSD and X-CoSD-E preserve the server LLM distribution, and experiments show that they significantly improve token generation speed while maintaining generation quality comparable to that of the server LLM.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-10 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.09166 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T07:24:00.861Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-10?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).