Metraipolis › archive › dc_2acdb2ffe377

HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving — arXiv:2608.11668

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.11668
License (as archived): CC0-1.0
Archived: 2026-10-19T05:31:07.964Z
SHA-256 of the archived text: ae911ce63b917c1d1355808d02240a47307542844d5097420d3eebc69d221455
Stable URL of this page: https://metraipolis.com/documento/dc_2acdb2ffe377

How to cite

HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving — arXiv:2608.11668 — arXiv (en), https://arxiv.org/abs/2608.11668 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2acdb2ffe377 (Metraipolis archive dc_2acdb2ffe377, sha256 ae911ce63b917c1d1355808d02240a47307542844d5097420d3eebc69d221455)

BibTeX

@misc{dc2acdb2ffe377,
  title = {HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving — arXiv:2608.11668},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.11668},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2acdb2ffe377; sha256 ae911ce63b917c1d1355808d02240a47307542844d5097420d3eebc69d221455},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.11668. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

A faster storage device should make serving faster. We find the opposite. High-Bandwidth Flash (HBF) stacks NAND behind a wide, package-local interface, promising flash-scale capacity with far lower read latency and higher bandwidth than an SSD. The obvious move is to keep an SSD-style Mooncake KV-offloading stack and swap in HBF underneath. We built that system and measured it: an extended TokenSim, four complete two-hour Qwen-Bailian production traces, five dense and mixture-of-experts models, and H100/B200 profiles. The upgrade backfires. Average end-to-end latency rises 2--5.5$\times$ and maximum SLO goodput falls 1.1--2.7$\times$ across H100 and B200, so the faster device yields a slower system. A cost-benefit model explains the paradox: a faster far tier pays off only when read I/O is the bottleneck, reads outweigh writes, and delivered bandwidth is sustainable. Transient KV violates all three at once. Buying flash through the package costs GPU near-tier capacity and bandwidth, while HBF's own read/write latency barely matters: scaling it 3.75$\times$ moves latency less than 1\%. Worse, the two-tier hierarchy keeps reuse in the near tier and hands HBF a relentless write-heavy stream. Writes outnumber reads on every trace, so a 3D-ICE model shows the stack hits its thermal limit well below peak bandwidth, and a TLC tier wears out sooner than the SSD pool it replaced. The device is fine; the drop-in deployment is not. HBF sucks as an SSD replacement for transient KV, but earns its place in LLM serving when used selectively with reuse-aware placement, write budgeting, and thermal coordination.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-13 in cs.AR).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2608.11668 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T05:16:12.067Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-08-13?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).