Metraipolis › archive › dc_2d977447699d

Accelerating LLM Decoding with In-DRAM Parallelism — arXiv:2611.02401

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.02401
License (as archived)
CC0-1.0
Archived
2026-11-03T18:28:26.357Z
SHA-256 of the archived text
87c809e1b9db3692864ee40524e1295be959be6b3d2486986ecb391da5b50122
Stable URL of this page
https://metraipolis.com/documento/dc_2d977447699d

How to cite

Accelerating LLM Decoding with In-DRAM Parallelism — arXiv:2611.02401 — arXiv (en), https://arxiv.org/abs/2611.02401 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d977447699d (Metraipolis archive dc_2d977447699d, sha256 87c809e1b9db3692864ee40524e1295be959be6b3d2486986ecb391da5b50122)
BibTeX
@misc{dc2d977447699d,
  title = {Accelerating LLM Decoding with In-DRAM Parallelism — arXiv:2611.02401},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.02401},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d977447699d; sha256 87c809e1b9db3692864ee40524e1295be959be6b3d2486986ecb391da5b50122},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02401. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Autoregressive decoding of large language models (LLMs) on edge devices is dominated by memory-bound general matrix--vector multiplication (GeMV) operations. Processing-using-DRAM (PUD) is a promising computing paradigm that can alleviate this bottleneck by reducing processor--memory data transfers and enabling massively parallel computation within DRAM (e.g., across 65,536 columns). However, weight matrices in individual LLM layers (e.g., a QKV matrix in Llama3 8B with 6,144 rows) are typically too small to fully utilize parallelism in PUD. To address this challenge, we propose an algorithm--system co-design for PUD-based LLM inference. At the algorithm level, we enable a subsequent projection to be packed with the current projection by reformulating it as a computation on the current residual state plus a correction for the residual update. Layer redundancy allows this correction to be approximated using fewer bit-serial steps. At the system level, we tailor data mapping and execution to improve utilization of either columns within DRAM subarrays (SIMD) or independently controlled banks (MIMD). Overall, we achieves up to $1.41\times$ end-to-end decoding speedup over the PUD baseline, corresponding to a $2.45\times$ speedup over processor-based execution.

Tema degli esploratori: calcolo e infrastrutture (elenco arXiv cs.AR, nuovi lavori del giorno).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2611.02401 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T18:10:37.666Z dalla pagina https://arxiv.org/list/cs.AR/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).