Metraipolis › archive › dc_a4bbea44b100

UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge — arXiv:2608.09291

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.09291
License (as archived)
CC0-1.0
Archived
2026-10-19T08:52:40.280Z
SHA-256 of the archived text
dda1da6f79d6db24d46cb558c97a2aec89d12344d9b06d90840293ea8794a942
Stable URL of this page
https://metraipolis.com/documento/dc_a4bbea44b100

How to cite

UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge — arXiv:2608.09291 — arXiv (en), https://arxiv.org/abs/2608.09291 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a4bbea44b100 (Metraipolis archive dc_a4bbea44b100, sha256 dda1da6f79d6db24d46cb558c97a2aec89d12344d9b06d90840293ea8794a942)
BibTeX
@misc{dca4bbea44b100,
  title = {UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge — arXiv:2608.09291},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09291},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a4bbea44b100; sha256 dda1da6f79d6db24d46cb558c97a2aec89d12344d9b06d90840293ea8794a942},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09291. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Edge LLM inference combines sparsity and low-bit quantization to meet device memory, latency, and power limits. Yet quantization shrinks weight payloads without proportionally reducing sparse metadata, so index traffic and nonzero extraction become critical SpMM bottlenecks. We introduce the Payload-to-Metadata Ratio (PMR) and show that improving PMR raises effective compute intensity in decoding. We present UnionSparse, an index-efficient framework that combines Index-Efficient Bitmap Encoding (IE-BME) with a SpMM kernel using Low-Bit Shared-Memory Parallel Decoding (LSPD). IE-BME amortizes metadata and aligns sparse traversal with fragment assembly, while LSPD improves small-batch execution. Under W4A4 quantization and 30%--70% sparsity, UnionSparse outperforms FlashLLM and SpInfer by 2.30x and 1.43x, and CUTLASS and cuBLAS Tensor Core by 1.56x and 3.46x, respectively. These results establish payload-extraction efficiency as a first-order concern for low-bit sparse inference on edge GPUs. Source code is available at: this https URL .

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-11 in cs.DC).
Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC).
Pagina del lavoro: https://arxiv.org/abs/2608.09291 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:50:23.750Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).