Metraipolis › archive › dc_4bdfdf25dbd0

Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference — arXiv:2610.39562

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.39562
License (as archived)
CC0-1.0
Archived
2026-10-31T06:13:18.723Z
SHA-256 of the archived text
325d40c2140a9600b5b7988f9cb774ca07a8fac71775019e4c5bfe2d774821b5
Stable URL of this page
https://metraipolis.com/documento/dc_4bdfdf25dbd0

How to cite

Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference — arXiv:2610.39562 — arXiv (en), https://arxiv.org/abs/2610.39562 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4bdfdf25dbd0 (Metraipolis archive dc_4bdfdf25dbd0, sha256 325d40c2140a9600b5b7988f9cb774ca07a8fac71775019e4c5bfe2d774821b5)
BibTeX
@misc{dc4bdfdf25dbd0,
  title = {Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference — arXiv:2610.39562},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.39562},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4bdfdf25dbd0; sha256 325d40c2140a9600b5b7988f9cb774ca07a8fac71775019e4c5bfe2d774821b5},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.39562. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-turn conversations with multimodal large language models (MLLMs) reuse input context, but long multimodal inputs increase time to first token (TTFT). Query-aware hard pruning reduces prefill tokens but permanently excludes information that later questions may require. We propose Raincheck, a framework for deferred multimodal prefill that decouples initial token selection from later information availability. An existing pruning method selects tokens for initial prefill; Raincheck preserves and progressively prefills unselected tokens in source order using unused compute capacity during decode. Unfinished prefill continues after answer return during available scheduling gaps. Attention visibility control and affine state mapping enable context updates after generation begins in full-attention and hybrid recurrent models. Prefill-decode fusion reduces repeated weight reads. We evaluate Qwen3-VL-8B and Qwen3.5-9B with 30% initial multimodal-token retention. Across both models and four pruning methods, Raincheck recovers on average 77.7% and 60.4% of the quality lost to matched hard pruning on MultiVerse and ConvBench, respectively. On MultiVerse, the SGLang implementation reduces mean first-turn TTFT by up to 24.1% relative to full prefill, with less than 0.5% increase in mean time per output token (TPOT).

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.39562 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T06:07:08.723Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).