Metraipolis › archive › dc_f350765ab4d2

PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX — arXiv:2608.17379

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.17379
License (as archived)
CC0-1.0
Archived
2026-10-18T14:33:30.726Z
SHA-256 of the archived text
f83d7ee860802150115f99bc5838fe3cf452584307609ca6a43f63e843f1ea24
Stable URL of this page
https://metraipolis.com/documento/dc_f350765ab4d2

How to cite

PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX — arXiv:2608.17379 — arXiv (en), https://arxiv.org/abs/2608.17379 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f350765ab4d2 (Metraipolis archive dc_f350765ab4d2, sha256 f83d7ee860802150115f99bc5838fe3cf452584307609ca6a43f63e843f1ea24)
BibTeX
@misc{dcf350765ab4d2,
  title = {PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX — arXiv:2608.17379},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.17379},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f350765ab4d2; sha256 f83d7ee860802150115f99bc5838fe3cf452584307609ca6a43f63e843f1ea24},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17379. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, whether selected target instructions execute at runtime, and speedup over frontier libraries across GEMM and attention workloads on H100 and B200 GPUs. Our evaluation shows that architecture-specific PTX capability remains uneven: success rates fall substantially on complex attention backward workloads, and executing the target instructions does not necessarily translate into competitive performance. No evaluated model consistently matches frontier libraries across the suite. We further adapt Qwen3.6-27B using supervised fine-tuning. Repair-conditioned training improves several tasks, but generalization remains uneven; data coverage, balance, and the quality of the reasoning teacher matter in addition to dataset size. PTXBench provides an auditable testbed for measuring and improving LLMs' ability to exploit evolving GPU architectures.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.17379 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T14:27:54.425Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).