Metraipolis › archive › dc_0b78178c2fdf

LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding — arXiv:2608.20530

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.20530
License (as archived)
CC0-1.0
Archived
2026-10-18T10:18:19.380Z
SHA-256 of the archived text
08b8c8c238a18fc512e3a20785166f388296f9263d8c99ab189d04e3188252ac
Stable URL of this page
https://metraipolis.com/documento/dc_0b78178c2fdf

How to cite

LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding — arXiv:2608.20530 — arXiv (en), https://arxiv.org/abs/2608.20530 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0b78178c2fdf (Metraipolis archive dc_0b78178c2fdf, sha256 08b8c8c238a18fc512e3a20785166f388296f9263d8c99ab189d04e3188252ac)
BibTeX
@misc{dc0b78178c2fdf,
  title = {LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding — arXiv:2608.20530},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.20530},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0b78178c2fdf; sha256 08b8c8c238a18fc512e3a20785166f388296f9263d8c99ab189d04e3188252ac},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.20530. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speculative decoding accelerates language-model inference by drafting future tokens the target model verifies in parallel. A diffusion-style drafter such as DFlash drafts an entire block in one forward pass. It is trained on the per-position marginals rather than on the joint distribution over the block, so the tokens it emits are individually plausible yet jointly incoherent. We introduce LiLiCorr, a Lightweight Likelihood-based model that Correlates the per-position marginals such a drafter produces. It keeps the top-K tokens at each position and processes them jointly, emitting an in and an out vector for each. Two candidates at consecutive positions match when the earlier out vector aligns, in cosine similarity, with the later in vector. Training scores the correct pairings highest and pushes competing ones down, so coherent blocks outscore incoherent ones. The joint distribution over the block, exponential in its length, is never materialized. One lightweight network pass produces all the vectors, the pairwise scores follow as batched matrix operations, leaving only a cheap greedy walk sequential. We co-train the DFlash drafter with LiLiCorr, so it proposes candidates that correlate into longer accepted sequences. Over the vanilla DFlash drafter it builds on, LiLiCorr accepts more and serves faster at all 72 settings we test: nine benchmarks at two target sizes under greedy and temperature-one decoding, plus a throughput sweep over six concurrencies, two input lengths and three output-entropy tiers. It raises acceptance length by 7 to 19%, while its single-pass scoring head costs only about 3% of the per-block latency. Against three concurrently developed methods that also restore coherence at draft time, all equally optimized on a common stack, LiLiCorr holds the highest throughput in 63 of those settings, ties within a measured noise floor in 6, and trails in only 3.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-24 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.20530 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:17:37.574Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).