Metraipolis › archive › dc_1bd18e1bbc86
StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference — arXiv:2611.08363
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08363
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-06T14:43:32.485Z
- SHA-256 of the archived text
fc4f8fe7dfd4b510dfbf9f980b648c572799e1d2db472fc81410c1bbb318c8ac- Stable URL of this page
https://metraipolis.com/documento/dc_1bd18e1bbc86
How to cite
StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference — arXiv:2611.08363 — arXiv (en), https://arxiv.org/abs/2611.08363 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1bd18e1bbc86 (Metraipolis archive dc_1bd18e1bbc86, sha256 fc4f8fe7dfd4b510dfbf9f980b648c572799e1d2db472fc81410c1bbb318c8ac)
BibTeX
@misc{dc1bd18e1bbc86,
title = {StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference — arXiv:2611.08363},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08363},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1bd18e1bbc86; sha256 fc4f8fe7dfd4b510dfbf9f980b648c572799e1d2db472fc81410c1bbb318c8ac},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08363. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Quantization reduces the memory and bandwidth demands of large language model (LLM) inference but can degrade accuracy; offloading higher-precision data can instead stall generation. We introduce StrataFlux, a framework that challenges this tradeoff by recovering accuracy lost to low-bit weight and key-value (KV) cache quantization while retaining quantization's memory and speed advantages. StrataFlux co-designs refinement representation, selection, and delivery to make host RAM and solid-state drives useful sources of additional precision. Its selective precision streaming reduces transfer volume, overlaps delivery with computation, and reuses fetched precision across decoding steps so generation need not wait for every transfer. We evaluate 2-4-bit weights and 2-3-bit KV-cache on dedicated GPUs and unified-memory platforms. For 2-bit Llama-3.1-8B-Instruct weights, StrataFlux reduces WikiText-2 perplexity from 9.46 to 8.40 while maintaining 95.4% of the quantized base's throughput on RTX 5090. With 2-bit weights and KV, StrataFlux raises Qwen3-8B's AIME 2024/2025 accuracy from 11.0% to 47.1% at unchanged GPU memory use. At 128K context, Llama-3.1-8B-Instruct with 3-bit weights and 2-bit KV achieves comparable perplexity and speed to vLLM's 4-bit AWQ weights and 8-bit KV on RTX 5090, using approximately half the GPU memory. These results show that GPU-resident precision need not set the accuracy ceiling, nor offloading-tier latency dictate decoding speed. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2611.08363 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-06T14:19:12.623Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).