Metraipolis › archive › dc_64a798d24a2a

AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding — arXiv:2608.02989

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.02989
License (as archived)
CC0-1.0
Archived
2026-10-31T12:31:22.231Z
SHA-256 of the archived text
2ec168c3fdd44e258fd634a81dbb9a3f206f9eb3c7b12e393f11a6441d8d616e
Stable URL of this page
https://metraipolis.com/documento/dc_64a798d24a2a

How to cite

AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding — arXiv:2608.02989 — arXiv (en), https://arxiv.org/abs/2608.02989 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_64a798d24a2a (Metraipolis archive dc_64a798d24a2a, sha256 2ec168c3fdd44e258fd634a81dbb9a3f206f9eb3c7b12e393f11a6441d8d616e)
BibTeX
@misc{dc64a798d24a2a,
  title = {AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding — arXiv:2608.02989},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.02989},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_64a798d24a2a; sha256 2ec168c3fdd44e258fd634a81dbb9a3f206f9eb3c7b12e393f11a6441d8d616e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.02989. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speculative decoding verifies a tree of draft tokens in one target-model forward pass. For a mixture-of-experts (MoE) target, however, parallel verification can activate the union of the experts selected by all tree nodes, even though only a small subset of those nodes reaches the accepted output. Token count, activated-expert union size, and expert-weight traffic are therefore distinct cost measures: reducing the token workload need not shrink the expert union proportionally, and under offloading, transfer traffic also depends on cache residency. We introduce AcceptMoE, a verifier-side expert selector that combines target-router scores with offline-estimated commitment probabilities and automatically adjusts the number of eligible experts for each verification block, eliminating the need for a user-specified expert budget. Under offloading, AcceptMoE conditions expert eligibility on cache residency instead of predicting natural routes and prefetching the corresponding expert weights. Although constraining target-expert eligibility changes the model distribution, across 12 model-task pairs spanning three MoE targets and four benchmarks, AcceptMoE's mean accuracy is 0.27 percentage points lower than that of EAGLE-3 speculative decoding with natural routing. Served with SGLang at batch size one, it reaches 1.290 times the throughput of this baseline with all expert weights in GPU memory, and 2.06 times under physical expert offloading, while reducing host-to-device traffic by 73.6 percent to 77.1 percent.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-05 in cs.CL).
Categorie arXiv: Machine Learning (cs.LG) ; Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2608.02989 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T12:17:37.802Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-05?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).