Metraipolis › archive › dc_8db624bb585d

SSR: Sparse Segment Reduction for Ternary GEMM Acceleration — arXiv:2610.08403

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.08403
License (as archived)
CC0-1.0
Archived
2026-10-07T11:41:47.539Z
SHA-256 of the archived text
b1011636c46b474a85f5ba9895ac0adeaf91a409ed933a677834fac61bbcdb8a
Stable URL of this page
https://metraipolis.com/documento/dc_8db624bb585d

How to cite

SSR: Sparse Segment Reduction for Ternary GEMM Acceleration — arXiv:2610.08403 — arXiv (en), https://arxiv.org/abs/2610.08403 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8db624bb585d (Metraipolis archive dc_8db624bb585d, sha256 b1011636c46b474a85f5ba9895ac0adeaf91a409ed933a677834fac61bbcdb8a)
BibTeX
@misc{dc8db624bb585d,
  title = {SSR: Sparse Segment Reduction for Ternary GEMM Acceleration — arXiv:2610.08403},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.08403},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8db624bb585d; sha256 b1011636c46b474a85f5ba9895ac0adeaf91a409ed933a677834fac61bbcdb8a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.08403. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) require substantial computational resources, limiting their deployment on resource-constrained hardware. Ternary LLMs mitigate these demands through weight quantization via ternary values, achieving significant compression often with 50-90% sparsity. However, existing approaches have limitations: methods optimized for ternary weights, such as BitNet, redundant segment reduction (RSR), and its improved version RSR++, do not exploit sparsity structures, while conventional sparse formats neglect ternary characteristics, foregoing dual optimization opportunities. In this paper, we introduce Sparse Segment Reduction (SSR), a ternary matrix multiplication method designed to accelerate the inference of ternary LLMs and general Ternary Weight Networks (TWNs). SSR has a dedicated optimized ternary data format and an algorithm that systematically exploits sparsity patterns through computation trees that scale with the sparsity. SSR provides theoretical gains with asymptotically faster inference than RSR++ for sparsity above 50%, while practical evaluations reveal performance improvements across all sparsity levels. Evaluation results show that SSR achieves 2.1-11.3x speedup over RSR++ on ternary GEMM with 45-95% sparsity. Furthermore, SSR achieves 3.5-6.3x end-to-end speedup and 4.9% of memory saving over RSR++ on the Llama-3 1B model inference.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.08403 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T11:19:24.830Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).