Metraipolis › archive › dc_cd59ce756481
ResServe: Accurate and Efficient FP4 LLM Inference with Residue Compensation — arXiv:2611.08713
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.08713
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-09T22:36:28.423Z
- SHA-256 of the archived text
f1e9ce42578ec25776e817b48b22a459e73c1105c3411e35444ba754507d43ee- Stable URL of this page
https://metraipolis.com/documento/dc_cd59ce756481
How to cite
ResServe: Accurate and Efficient FP4 LLM Inference with Residue Compensation — arXiv:2611.08713 — arXiv (en), https://arxiv.org/abs/2611.08713 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cd59ce756481 (Metraipolis archive dc_cd59ce756481, sha256 f1e9ce42578ec25776e817b48b22a459e73c1105c3411e35444ba754507d43ee)
BibTeX
@misc{dccd59ce756481,
title = {ResServe: Accurate and Efficient FP4 LLM Inference with Residue Compensation — arXiv:2611.08713},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.08713},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cd59ce756481; sha256 f1e9ce42578ec25776e817b48b22a459e73c1105c3411e35444ba754507d43ee},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.08713. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Block-scaled 4-bit floating-point (FP4) quantization reduces large language model inference costs but can degrade task quality. Higher-precision computation can mitigate this loss at the expense of some of the efficiency gains from FP4 inference. Narrow matrix computations common in decoding often leave spare hardware capacity for low-cost precision recovery. ResServe combines an FP4 activation base with an FP4-quantized residue representing information lost during quantization. It uses this spare capacity to compute the residue's contribution and combines results on-chip, avoiding extra result transfers. We apply this principle to linear layers and decode attention, while selective compensation limits added work when processing input prompts. On NVIDIA B200, compensated linear kernels using the NVFP4 format improve numerical precision while retaining latency close to uncompensated execution for the evaluated operations with few rows. In the evaluated NVFP4 inference configurations, ResServe adds at most 3.4% request-latency overhead over W4A4 and reduces request latency by 22.4-53.3% relative to W4A8. Across five models, code-generation accuracy improves over same-format uncompensated inference, with mean relative gains of 15.04% for NVFP4 and 53.57% for MXFP4. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.DC, nuovi lavori del giorno). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC). Pagina del lavoro: https://arxiv.org/abs/2611.08713 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-09T22:31:17.034Z dalla pagina https://arxiv.org/list/cs.DC/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).