Metraipolis › archive › dc_977003cdaf55

HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding — arXiv:2610.21436

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.21436
License (as archived): CC0-1.0
Archived: 2026-10-16T09:57:41.695Z
SHA-256 of the archived text: cbab53750b4eb5226ddaf07b2061ae3150074c514ec87863325c230eda7d433e
Stable URL of this page: https://metraipolis.com/documento/dc_977003cdaf55

How to cite

HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding — arXiv:2610.21436 — arXiv (en), https://arxiv.org/abs/2610.21436 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_977003cdaf55 (Metraipolis archive dc_977003cdaf55, sha256 cbab53750b4eb5226ddaf07b2061ae3150074c514ec87863325c230eda7d433e)

BibTeX

@misc{dc977003cdaf55,
  title = {HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding — arXiv:2610.21436},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21436},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_977003cdaf55; sha256 cbab53750b4eb5226ddaf07b2061ae3150074c514ec87863325c230eda7d433e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21436. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Device-cloud speculative decoding reduces server computation, but growing a deep draft tree with a small language model (SLM) requires repeated passes on a mobile neural processing unit (NPU). We present HyCo, which assigns depth to the server model's native multi-token prediction (MTP) head, executed on the device, and alternative branches to one SLM pass. The server returns the hidden state, input embedding, and key-value entries needed by the head instead of executing its sequential steps. Quantization, a restricted vocabulary, and a 1K-token window fit both drafters to the NPU. An {\epsilon}-threshold rule additionally relaxes exact preservation of the verifier's sampling distribution. In single-client runs with a Galaxy S26 Ultra over LTE/5G, HyCo reduces server GPU time per output token relative to server-side MTP self-speculation by 38.6% on Spec-Bench and 29.2% on 32K-token summarization. With the same acceptance rule, three Spec-Bench runs per system show 9.0 percentage points more saving and 1.9x delivery speed than an SLM-only drafter. HyCo differs from Cloud+MTP by +0.2 judge-score points (paired 95% CI: -0.9 to +1.3); exploratory translation favors SpecEdge-{\epsilon}. Without relaxation, HyCo exact saves 27.7% and 11.9% on the two datasets.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.21436 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:44:47.652Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).