Metraipolis › archive › dc_fa6aa3ba9ec2

HBFlex: A Flexible Memory System for Bridging Fine-Grained LLM States and Coarse-Grained HBF Parallel Execution — arXiv:2609.18675

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.18675
License (as archived)
CC0-1.0
Archived
2026-10-15T13:58:01.509Z
SHA-256 of the archived text
7ed9bbcfc86f030c7ed9fc24a1a02d6287a5d7ba5b93debcbcd36defb8624a24
Stable URL of this page
https://metraipolis.com/documento/dc_fa6aa3ba9ec2

How to cite

HBFlex: A Flexible Memory System for Bridging Fine-Grained LLM States and Coarse-Grained HBF Parallel Execution — arXiv:2609.18675 — arXiv (en), https://arxiv.org/abs/2609.18675 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_fa6aa3ba9ec2 (Metraipolis archive dc_fa6aa3ba9ec2, sha256 7ed9bbcfc86f030c7ed9fc24a1a02d6287a5d7ba5b93debcbcd36defb8624a24)
BibTeX
@misc{dcfa6aa3ba9ec2,
  title = {HBFlex: A Flexible Memory System for Bridging Fine-Grained LLM States and Coarse-Grained HBF Parallel Execution — arXiv:2609.18675},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.18675},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_fa6aa3ba9ec2; sha256 7ed9bbcfc86f030c7ed9fc24a1a02d6287a5d7ba5b93debcbcd36defb8624a24},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18675. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models (LLMs) require increasing memory capacity to accommodate growing model weights and KV caches. High-Bandwidth Flash (HBF) offers high memory density and aggregate read bandwidth through massive plane-level parallelism, making it an attractive option for LLM serving. However, serving LLMs entirely from HBF introduces three challenges: fine-grained KV reads create placement and access imbalance, incremental writes interfere with foreground reads, and mixed KV lifetimes amplify garbage collection. Hybrid HBM/HBF designs retain HBM to support dynamic KV management, but this allocation reduces the HBF resources available under a fixed packaging budget, limiting aggregate HBF bandwidth. We present HBFlex, a full-HBF memory system with coordinated optimizations for KV reads, writes, and reclamation. HBFlex balances KV placement and attention accesses to improve plane utilization. It aggregates incremental updates and schedules writeback within sufficiently long compute windows to reduce write--read interference. It also combines lifetime-guided block packing with deferred reclamation to reduce valid-page migration. We evaluate HBFlex through trace-driven simulation across different configurations. HBFlex achieves average throughput speedups of up to 1.58$\times$ over FlashAccel and 3.30$\times$ over H3, benefiting from higher HBF bandwidth and more efficient management of dynamic KV-cache reads, writes, and erases.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-17 in cs.AR).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2609.18675 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:49:21.819Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).