Metraipolis › archive › dc_7f2cd8311a0d

PATTON: Enabling Commodity PIM for Production LLM Serving — arXiv:2609.11392

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.11392
License (as archived)
CC0-1.0
Archived
2026-10-16T05:44:56.924Z
SHA-256 of the archived text
23c73eb6e38f164371d1142be8eaa49f8eb815a2ebba9a182060bd4eba352a41
Stable URL of this page
https://metraipolis.com/documento/dc_7f2cd8311a0d

How to cite

PATTON: Enabling Commodity PIM for Production LLM Serving — arXiv:2609.11392 — arXiv (en), https://arxiv.org/abs/2609.11392 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_7f2cd8311a0d (Metraipolis archive dc_7f2cd8311a0d, sha256 23c73eb6e38f164371d1142be8eaa49f8eb815a2ebba9a182060bd4eba352a41)
BibTeX
@misc{dc7f2cd8311a0d,
  title = {PATTON: Enabling Commodity PIM for Production LLM Serving — arXiv:2609.11392},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.11392},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_7f2cd8311a0d; sha256 23c73eb6e38f164371d1142be8eaa49f8eb815a2ebba9a182060bd4eba352a41},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11392. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Processing-in-Memory (PIM) is promising for accelerating memory-bound decode attention, but attention acceleration alone is insufficient for production LLM serving, where engines dynamically allocate, populate, share, cache, and reclaim logical KV cache blocks. Supporting this lifecycle on commodity PIM requires efficient physical memory allocation, block-to-address mapping, and command generation. For the Value cache, these requirements create a fundamental conflict among GEMV efficiency, single-token write efficiency, and memory capacity: GEMV-optimized layouts scatter newly generated Value vectors across rows, making writes costly, while finer-grained memory sharing improves capacity utilization but fragments GEMV reductions. We present PATTON, a PIM runtime that integrates production LLM serving engines with commodity PIM. PATTON introduces hierarchical granule allocation: block-sized Key and Value granules map one-to-one to logical token blocks, fixing their physical placements and commands, while coarser granules group blocks for efficient GEMV execution and memory utilization. A Commit Zone stages partial Value blocks for efficient single-token writes before committing them to GEMV-optimized locations. PATTON tracks these placements to generate KV cache writes and QK-transpose/SV commands. Across attention execution and runtime-induced prefill recomputation, PATTON achieves an average 1.95x speedup and 4.83x higher energy efficiency over evaluated baselines, requires no PIM processing-unit modifications, and maintains a KV cache hit rate comparable to the native GPU KV cache in vLLM.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-11 in cs.AR).
Categorie arXiv: Hardware Architecture (cs.AR).
Pagina del lavoro: https://arxiv.org/abs/2609.11392 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T05:26:59.340Z dalla pagina https://arxiv.org/catchup/cs.AR/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).