Metraipolis › archive › dc_4d377cd2c2e1
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention — arXiv:2609.13134
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.13134
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T03:03:02.432Z
- SHA-256 of the archived text
5c26dcc3c27511c3dd0b7dc9261ceb67804ed263ed8491e4915911845308ee50- Stable URL of this page
https://metraipolis.com/documento/dc_4d377cd2c2e1
How to cite
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention — arXiv:2609.13134 — arXiv (en), https://arxiv.org/abs/2609.13134 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4d377cd2c2e1 (Metraipolis archive dc_4d377cd2c2e1, sha256 5c26dcc3c27511c3dd0b7dc9261ceb67804ed263ed8491e4915911845308ee50)
BibTeX
@misc{dc4d377cd2c2e1,
title = {Rethinking Heterogeneous System Disaggregation for Subquadratic Attention — arXiv:2609.13134},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.13134},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4d377cd2c2e1; sha256 5c26dcc3c27511c3dd0b7dc9261ceb67804ed263ed8491e4915911845308ee50},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13134. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Frontier language models are more aggressively using subquadratic attention to reduce the memory footprint and compute requirements during inference while still delivering frontier accuracy. While existing systems make dense attention-centric disaggregated serving decisions, we show that disaggregating inference around the unique arithmetic intensity and memory footprint of subquadratic attention LLMs can achieve significant throughput and energy efficiency gains on emerging DRAM-based and SRAM-only heterogeneous systems. We introduce SQD (SubQuadratic Disaggregation), a fine-grained heterogeneous disaggregation scheme that splits decode by quadratic and subquadratic attention rather than by operator type, and that applies across subquadratic attention variants. For sparse attention LLMs, we disaggregate decode into top-k selection, which must index through the full KV, and top-k attention plus FFN, which have static memory footprints. For linear and sliding-window attention LLMs, we disaggregate decode into dense attention layers and subquadratic attention layers plus FFN. In an adjusted 8xB200 heterogeneous system proxy, we observe average tokens/J improvements of 53% on GLM 5.2, 31% on Nemotron 3 Ultra, and 56% on Gemma 4 31B over the strongest GPU-only baselines. In an analytical model of a Rubin plus LPX system with fixed power budgets, we observe 1.2x to 1.5x tighter achievable latencies and up to 3.6x higher throughput over the best baseline of attention-FFN disaggregation. Our experiments also reveal architectural insights on chip and interconnect provisioning for next-generation heterogeneous systems serving subquadratic attention. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2609.13134 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T02:51:27.839Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).