Metraipolis › archive › dc_3fa40462b359

FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding — arXiv:2610.20392

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.20392
License (as archived)
CC0-1.0
Archived
2026-10-15T14:23:51.170Z
SHA-256 of the archived text
6eb30adafe3ebd32a8020d94c4f804bd61488b37d60b27aa092e6ff0ccaac617
Stable URL of this page
https://metraipolis.com/documento/dc_3fa40462b359

How to cite

FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding — arXiv:2610.20392 — arXiv (en), https://arxiv.org/abs/2610.20392 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3fa40462b359 (Metraipolis archive dc_3fa40462b359, sha256 6eb30adafe3ebd32a8020d94c4f804bd61488b37d60b27aa092e6ff0ccaac617)
BibTeX
@misc{dc3fa40462b359,
  title = {FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding — arXiv:2610.20392},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.20392},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3fa40462b359; sha256 6eb30adafe3ebd32a8020d94c4f804bd61488b37d60b27aa092e6ff0ccaac617},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.20392. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speculative decoding accelerates LLM generation through lightweight drafting and parallel verification. It is particularly promising for long inputs, where KV-cache access costs can be amortized over multiple accepted tokens. Yet most drafters are trained on short sequences and degrade on long inputs, while long-context training is costly. Existing training-free approaches restrict the context window, rescale positions, or augment recent context with retrieved distant entries. Drafters, however, read target hidden states that already encode contextual and positional information, and their lightweight architecture makes full-history attention affordable. We propose FOLD, a training-free method that preserves exact local positions and treats all distant entries as far memory at a shared position. To counter attention dilution as the memory grows, we sharpen attention within it to emphasize relevant entries while separately controlling its total weight relative to the local window. Across six target-drafter pairs and diverse benchmarks of up to 256K tokens, FOLD predominantly delivers the highest speedup, with the best case more than doubling the speedup over autoregressive decoding, from 2.08$\times$ to 4.22$\times$.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.20392 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:53:35.963Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).