Metraipolis › archive › dc_ce9f2f2a4d00
Elastic Precision: Software-Hardware Co-Design of a Mixed-Precision Architecture for Long-Context LLM Inference — arXiv:2611.00915
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00915
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T11:12:59.325Z
- SHA-256 of the archived text
98c522c7876e63a232e5bde135063e059be843b8e57e3b114ea65c01900eba46- Stable URL of this page
https://metraipolis.com/documento/dc_ce9f2f2a4d00
How to cite
Elastic Precision: Software-Hardware Co-Design of a Mixed-Precision Architecture for Long-Context LLM Inference — arXiv:2611.00915 — arXiv (en), https://arxiv.org/abs/2611.00915 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ce9f2f2a4d00 (Metraipolis archive dc_ce9f2f2a4d00, sha256 98c522c7876e63a232e5bde135063e059be843b8e57e3b114ea65c01900eba46)
BibTeX
@misc{dcce9f2f2a4d00,
title = {Elastic Precision: Software-Hardware Co-Design of a Mixed-Precision Architecture for Long-Context LLM Inference — arXiv:2611.00915},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00915},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ce9f2f2a4d00; sha256 98c522c7876e63a232e5bde135063e059be843b8e57e3b114ea65c01900eba46},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00915. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Low-bit quantization reduces the data movement and computation costs of large language model (LLM) inference, but uniform widths can waste precision and degrade model quality. Existing mixed-precision methods do not fully account for how the other input of the current GEMM tile affects output error when selecting one input's width. Using both inputs to guide precision introduces additional estimation and selection overheads on conventional hardware. We propose Coffee, a runtime precision allocation architecture for long-context LLM inference. Under a fixed average bit-width budget, Coffee combines statistics from both current input matrices to estimate output errors at candidate widths and select each subarray's two input widths, allocating more bits where they yield greater estimated error reductions. Its mixed-precision hardware supports the selected widths in data access and computation, while overlapping error estimation and width selection with transfers and computation to limit stalls. The same read path supports attention-guided KV reads during decoding, reducing memory traffic. Coffee achieves lower perplexity and higher average zero-shot accuracy than the evaluated quantization baselines, and higher long-context retrieval recall than PLENA. For long-context decoding, system modeling projects up to $2.55\times$ the throughput and $2.61\times$ the energy efficiency of the reconfigured PLENA baseline, and up to $4.96\times$ and $12.56\times$, respectively, compared with the B200 GPU baseline. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno). Categorie arXiv: Hardware Architecture (cs.AR). Pagina del lavoro: https://arxiv.org/abs/2611.00915 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T11:00:22.165Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).