Metraipolis › archive › dc_f350765ab4d2
PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX — arXiv:2608.17379
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.17379
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-18T14:33:30.726Z
- SHA-256 of the archived text
f83d7ee860802150115f99bc5838fe3cf452584307609ca6a43f63e843f1ea24- Stable URL of this page
https://metraipolis.com/documento/dc_f350765ab4d2
How to cite
PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX — arXiv:2608.17379 — arXiv (en), https://arxiv.org/abs/2608.17379 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f350765ab4d2 (Metraipolis archive dc_f350765ab4d2, sha256 f83d7ee860802150115f99bc5838fe3cf452584307609ca6a43f63e843f1ea24)
BibTeX
@misc{dcf350765ab4d2,
title = {PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX — arXiv:2608.17379},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.17379},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f350765ab4d2; sha256 f83d7ee860802150115f99bc5838fe3cf452584307609ca6a43f63e843f1ea24},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.17379. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, whether selected target instructions execute at runtime, and speedup over frontier libraries across GEMM and attention workloads on H100 and B200 GPUs. Our evaluation shows that architecture-specific PTX capability remains uneven: success rates fall substantially on complex attention backward workloads, and executing the target instructions does not necessarily translate into competitive performance. No evaluated model consistently matches frontier libraries across the suite. We further adapt Qwen3.6-27B using supervised fine-tuning. Repair-conditioned training improves several tasks, but generalization remains uneven; data coverage, balance, and the quality of the reasoning teacher matter in addition to dataset size. PTXBench provides an auditable testbed for measuring and improving LLMs' ability to exploit evolving GPU architectures. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-19 in cs.AI). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2608.17379 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-18T14:27:54.425Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-19?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).