Metraipolis › archive › dc_a4ad8eb5e95a

BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs — arXiv:2611.00547

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.00547
License (as archived)
CC0-1.0
Archived
2026-11-03T11:12:57.912Z
SHA-256 of the archived text
01078918fab30b9dcfa4407e21d2d44ec2b84a0ff7e512b4536d69c656f48957
Stable URL of this page
https://metraipolis.com/documento/dc_a4ad8eb5e95a

How to cite

BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs — arXiv:2611.00547 — arXiv (en), https://arxiv.org/abs/2611.00547 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a4ad8eb5e95a (Metraipolis archive dc_a4ad8eb5e95a, sha256 01078918fab30b9dcfa4407e21d2d44ec2b84a0ff7e512b4536d69c656f48957)
BibTeX
@misc{dca4ad8eb5e95a,
  title = {BitLLM: Efficient Low-Bit, Conversion-Hiding LLM Decoding on Mobile GPUs — arXiv:2611.00547},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.00547},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a4ad8eb5e95a; sha256 01078918fab30b9dcfa4407e21d2d44ec2b84a0ff7e512b4536d69c656f48957},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.00547. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The rapid scaling of large language models (LLMs) has unlocked remarkable capabilities across various domains. However, deploying LLMs on mobile devices remains challenging because of their high computational and memory requirements. Low-bit quantization (e.g., 4-bit weights, 8-bit activations) can reduce model size while preserving accuracy, but its practical efficacy on mobile GPUs has been limited so far. In this paper, we identify that the performance gap stems primarily from frequent data type conversions between quantized and floating-point values, and dynamic and inefficient memory access patterns caused by the growing Key-Value (KV) cache during autoregressive decoding. We introduce BitLLM, a mobile inference engine that addresses type conversion overhead and suboptimal memory accesses in quantized LLMs. Our approach features an adaptive layout and packing strategy for the KV cache that ensures efficient memory access throughout LLM decoding. This is coupled with an integer-native execution pipeline that defers and overlaps type conversion outside the compute hot path, keeping the critical MatMul stages in the integer domain. Implemented on off-the-shelf mobile GPUs, BitLLM outperforms leading engines, i.e., this http URL , MNN-LLM, LiteRT-LM, and MLC-LLM, achieving average speedups of 1.78$\times$, 1.73$\times$, 1.58$\times$, and 1.47$\times$, respectively, across various models and quantization levels. Under long-context generation, BitLLM reduces average energy consumption (per token) by up to 88.9%.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR) ; Machine Learning (cs.LG); Operating Systems (cs.OS); Performance (cs.PF).
Pagina del lavoro: https://arxiv.org/abs/2611.00547 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T11:00:22.162Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).