Metraipolis › archive › dc_e8f6bcbafdab
BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs — arXiv:2611.00547
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.00547
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:27:54.799Z
- SHA-256 of the archived text
9ec349d21f9ba8df4f398ed0d40ee1f69932a32f1f737b1a1715b9691d7c0066- Stable URL of this page
https://metraipolis.com/documento/dc_e8f6bcbafdab
How to cite
BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs — arXiv:2611.00547 — arXiv (en), https://arxiv.org/abs/2611.00547 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e8f6bcbafdab (Metraipolis archive dc_e8f6bcbafdab, sha256 9ec349d21f9ba8df4f398ed0d40ee1f69932a32f1f737b1a1715b9691d7c0066)
BibTeX
@misc{dce8f6bcbafdab,
title = {BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs — arXiv:2611.00547},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.00547},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e8f6bcbafdab; sha256 9ec349d21f9ba8df4f398ed0d40ee1f69932a32f1f737b1a1715b9691d7c0066},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00547. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
The rapid scaling of large language models (LLMs) has unlocked remarkable capabilities across various domains. However, deploying LLMs on mobile devices remains challenging because of their high computational and memory requirements. Low-bit quantization (e.g., 4-bit weights, 8-bit activations) can reduce model size while preserving accuracy, but its practical efficacy on mobile GPUs has been limited so far. In this paper, we identify that the performance gap stems primarily from frequent data type conversions between quantized and floating-point values, and dynamic and inefficient memory access patterns caused by the growing Key-Value (KV) cache during autoregressive decoding. We introduce BitLLM, a mobile inference engine that addresses type conversion overhead and suboptimal memory accesses in quantized LLMs. Our approach features an adaptive layout and packing strategy for the KV cache that ensures efficient memory access throughout LLM decoding. This is coupled with an integer-native execution pipeline that defers and overlaps type conversion outside the compute hot path, keeping the critical MatMul stages in the integer domain. Implemented on off-the-shelf mobile GPUs, BitLLM outperforms leading engines, i.e., this http URL , MNN-LLM, LiteRT-LM, and MLC-LLM, achieving average speedups of 1.78$\times$, 1.73$\times$, 1.58$\times$, and 1.47$\times$, respectively, across various models and quantization levels. Under long-context generation, BitLLM reduces average energy consumption (per token) by up to 88.9%. Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno). Categorie arXiv: Hardware Architecture (cs.AR) ; Machine Learning (cs.LG); Operating Systems (cs.OS); Performance (cs.PF). Pagina del lavoro: https://arxiv.org/abs/2611.00547 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:10:37.655Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).