Metraipolis › archive › dc_4114d58ffd08
OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming — arXiv:2609.34653
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34653
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T20:58:02.091Z
- SHA-256 of the archived text
6014909290f7bd1b52062c77dc12dfddf2edc6fca15d28772e02835d7c724347- Stable URL of this page
https://metraipolis.com/documento/dc_4114d58ffd08
How to cite
OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming — arXiv:2609.34653 — arXiv (en), https://arxiv.org/abs/2609.34653 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4114d58ffd08 (Metraipolis archive dc_4114d58ffd08, sha256 6014909290f7bd1b52062c77dc12dfddf2edc6fca15d28772e02835d7c724347)
BibTeX
@misc{dc4114d58ffd08,
title = {OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming — arXiv:2609.34653},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34653},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4114d58ffd08; sha256 6014909290f7bd1b52062c77dc12dfddf2edc6fca15d28772e02835d7c724347},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34653. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
On-device streaming omni-modal inference safeguards user privacy and eliminates prohibitive per-token API costs, but faces a critical bottleneck: the continuous influx of multimodal data rapidly exhausts constrained memory and compute budgets via monotonic KV cache growth. Existing sparse attention methods fall short, either incurring prohibitive online estimation latency or destroying interleaved cross-modal context, while failing to resolve physical memory fragmentation. We present OmniTide, the first algorithm-system co-design tailored for efficient on-device streaming omni-modal inference. Driven by the observation of modality-aware structural sparsity, OmniTide adopts a unit-based abstraction with two components: (1) At the algorithm level, OmniPick logically retains critical multimodal context based on unit boundaries and modality importance to preserve task accuracy; (2) At the system level, OmniPage physically partitions the cache by retention likelihood and dynamically compacts surviving sparse tokens, minimizing both memory fragmentation and data-movement overhead. Extensive evaluations across three streaming benchmarks and two consumer-device architectures show that OmniTide achieves up to $12.72\times$ kernel speedups and $2.40\times$ lower stream-loop latency. On StreamingBench, it improves accuracy by up to 18.0 percentage points over sliding-window baselines at comparable session cost. OmniPage further reduces the physical KV span by up to 26.7% relative to native logical eviction, unlocking real-time, infinite-context streaming on edge devices. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.34653 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T20:29:01.621Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).