Metraipolis › archive › dc_38ac23edecb3
Disaggregated Quantization: Specializing LLM Prefill and Decode — arXiv:2609.26333
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.26333
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T15:07:57.522Z
- SHA-256 of the archived text
6b2afefee4ef2fe8a2640cf2f10b680ce47fb859affc731202f80ec2541bdda3- Stable URL of this page
https://metraipolis.com/documento/dc_38ac23edecb3
How to cite
Disaggregated Quantization: Specializing LLM Prefill and Decode — arXiv:2609.26333 — arXiv (en), https://arxiv.org/abs/2609.26333 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_38ac23edecb3 (Metraipolis archive dc_38ac23edecb3, sha256 6b2afefee4ef2fe8a2640cf2f10b680ce47fb859affc731202f80ec2541bdda3)
BibTeX
@misc{dc38ac23edecb3,
title = {Disaggregated Quantization: Specializing LLM Prefill and Decode — arXiv:2609.26333},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.26333},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_38ac23edecb3; sha256 6b2afefee4ef2fe8a2640cf2f10b680ce47fb859affc731202f80ec2541bdda3},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.26333. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Prefill and decode reward different approaches to quantization: low-precision arithmetic accelerates prompt processing, while compact weights reduce memory traffic during generation. We propose "disaggregated quantization" (DQ), which specializes computation formats, weights and storage placement to both of these phases. On Qwen 3 and Gemma 3, removing activation quantization specifically on decode improves accuracy on decode-heavy tasks without increasing inference cost. Training separate compute-native prefill weights accelerates prompt processing relative to weight-only inference while matching or exceeding its accuracy at 2-3-bit decode on both decode-heavy and prefill-heavy tasks. With released Qwen3.8-27B GGUF decoders, training an NVFP4 prefiller improves 1-bit accuracy by 32.5 points on MMLU-Pro and 35.3 on MMMU-Pro without modifying the decode checkpoint. To accommodate the additional checkpoint on a single device, offloaded disaggregated prefill (ODP) streams its weights from SSD, amortizing loading over prompt length. On the same 27B model, ODP delivers a 1.78x time-to-first-token speedup over the weight-only baseline at 8K prompt length in this http URL . We evaluate accuracy under disaggregated serving in vLLM and further validate shared-weight format disaggregation through post-training quantization on models up to 2.8T parameters. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.26333 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T14:37:29.484Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).