Metraipolis › archive › dc_30b2aec52f7d

On Design Principles for Efficient Heterogeneous DRAM-PIM-GPU Systems — arXiv:2608.04169

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.04169
License (as archived)
CC0-1.0
Archived
2026-10-31T11:25:40.464Z
SHA-256 of the archived text
15a4b0b5a6e62c56c42dc37bad990281c939fe2c55e47c351d2e52ad317f8072
Stable URL of this page
https://metraipolis.com/documento/dc_30b2aec52f7d

How to cite

On Design Principles for Efficient Heterogeneous DRAM-PIM-GPU Systems — arXiv:2608.04169 — arXiv (en), https://arxiv.org/abs/2608.04169 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_30b2aec52f7d (Metraipolis archive dc_30b2aec52f7d, sha256 15a4b0b5a6e62c56c42dc37bad990281c939fe2c55e47c351d2e52ad317f8072)
BibTeX
@misc{dc30b2aec52f7d,
  title = {On Design Principles for Efficient Heterogeneous DRAM-PIM-GPU Systems — arXiv:2608.04169},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.04169},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_30b2aec52f7d; sha256 15a4b0b5a6e62c56c42dc37bad990281c939fe2c55e47c351d2e52ad317f8072},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.04169. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Heterogeneous DRAM-based processing-in-memory (PIM)-GPU systems promise significant efficiency gains for decode-phase large language model (LLM) inference, particularly in long-output generation, yet current design practices overlook critical factors that determine real-world performance. Through systematic evaluation of diverse architectures and workloads (OPT-7B/70B, Mamba2-2.7B/70B), we reveal three fundamental design principles: (i) static power consumption (DRAM leakage, refresh, and GPU idle power) can dominate the efficiency calculus, causing dynamic-only models to overestimate tokens/s/W by up to 3.85X for realistic deployments (Mamba2-2.7B, batch size 1, 128 input tokens, and 2,048 output tokens); (ii) decoding performance is monotonically non-decreasing with channel count across all evaluated models and workloads, generally plateauing at high channel counts for low-batch workloads; under a fixed-capacity sweep, all models instead share a common near-optimal hierarchy configuration, with substantially larger misconfiguration penalties for attention-based models; (iii) workload mapping strategies provide bounded improvements (up to 14.0%/17.4% kernel-level latency/energy reduction, up to 5.6% end-to-end gain) and are not primary bottlenecks. Significant efficiency gains require system-wide co-optimization. These principles provide design-space guidance for architects designing the next generation of memory-accelerated LLM systems.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-06 in cs.AR).
Categorie arXiv: Hardware Architecture (cs.AR) ; Emerging Technologies (cs.ET).
Pagina del lavoro: https://arxiv.org/abs/2608.04169 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-31T11:14:40.700Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-08-06?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).