Metraipolis › archive › dc_ae643dcd1ec0

TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding — arXiv:2608.26112

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.26112
License (as archived): CC0-1.0
Archived: 2026-10-17T15:17:58.922Z
SHA-256 of the archived text: e0e87a0a85455de8d87ec6c248397f3aea9777f4bc738b31e30c97c535a65123
Stable URL of this page: https://metraipolis.com/documento/dc_ae643dcd1ec0

How to cite

TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding — arXiv:2608.26112 — arXiv (en), https://arxiv.org/abs/2608.26112 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ae643dcd1ec0 (Metraipolis archive dc_ae643dcd1ec0, sha256 e0e87a0a85455de8d87ec6c248397f3aea9777f4bc738b31e30c97c535a65123)

BibTeX

@misc{dcae643dcd1ec0,
  title = {TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding — arXiv:2608.26112},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.26112},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ae643dcd1ec0; sha256 e0e87a0a85455de8d87ec6c248397f3aea9777f4bc738b31e30c97c535a65123},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.26112. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve inference by organizing proposals into multiple candidate paths, increasing the accepted length. However, existing tree-structured methods use a single drafter for all drafting steps, creating a dilemma: a smaller drafter is fast but yields lower-quality trees, whereas a larger drafter improves tree quality but suffers from high latency. To address this, we propose TreeGraft, a multi-drafter framework in which drafters of different costs jointly construct a shared draft tree. TreeGraft uses the stronger drafter to rescore candidates by updating scores assigned by the weaker drafter, reselect grafting positions, and recover promising paths left unexplored. It also integrates stronger drafter expansions non-destructively, preserving existing branches that may still be accepted by the target model. Together, these designs improve the quality of the shared draft tree. To control the drafting cost, TreeGraft introduces a lightweight scheduler distilled from an offline value system to decide when to call the stronger drafter. Across 10 model pairs and 6 benchmarks, TreeGraft outperforms the better of the two fixed single-drafter endpoint strategies by 15.1% on average, reaching a maximum gain of 26.6%. Our code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-28 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.26112 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T15:15:18.007Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).