Metraipolis › archive › dc_dec03958894f
Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler Enforcement, and Static Verification — arXiv:2609.11356
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.11356
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T05:44:51.723Z
- SHA-256 of the archived text
8c53e5f1a2a151db73a9330d9a447cc788c30d52b989edd0c97f8b38802230eb- Stable URL of this page
https://metraipolis.com/documento/dc_dec03958894f
How to cite
Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler Enforcement, and Static Verification — arXiv:2609.11356 — arXiv (en), https://arxiv.org/abs/2609.11356 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dec03958894f (Metraipolis archive dc_dec03958894f, sha256 8c53e5f1a2a151db73a9330d9a447cc788c30d52b989edd0c97f8b38802230eb)
BibTeX
@misc{dcdec03958894f,
title = {Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler Enforcement, and Static Verification — arXiv:2609.11356},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.11356},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dec03958894f; sha256 8c53e5f1a2a151db73a9330d9a447cc788c30d52b989edd0c97f8b38802230eb},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.11356. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Determinism and numerical reproducibility are increasingly required of GPU kernels in machine learning systems, yet deterministic implementations of the same kernel can still differ bit for bit. Floating-point reduction order is the primary cause, alongside partial-sum precision, fused multiply-add operations, and rounding placement. These choices may be hand-coded, selected by a block-level language such as Triton, or hidden inside a closed-source library such as cuBLAS or rocBLAS. A tile shape chosen for performance therefore also determines the arithmetic, potentially breaking batch invariance. Preserving a fixed order can cost up to 20 percent, while an autotuner cannot identify which configurations are bitwise equivalent. We characterize the factors determining the bitwise behavior of reductions and general matrix multiplication (GEMM). First, we introduce a descriptor of GEMM reduction order, including the partitioning of K in split-K GEMM. Using it, we perform the first black-box reconstruction of a closed-source library's arithmetic for bit-level correctness. Our family of Triton GEMMs matches NVIDIA cuBLAS in all tested cases on Blackwell and Hopper. For realistic LLM shapes with fused epilogues, it matches or exceeds this http URL performance. Second, we enforce balanced-tree reduction during Triton lowering and introduce a data-layout optimization that brings 19 of 27 kernels on GB300 and H100 within 10 percent of free-order performance. Third, we develop sound static checkers for bitwise equivalence between compiled GPU kernels, including the first checker spanning NVIDIA PTX and AMD GCN. Integrated into Triton's autotuner, the checker restricts search to a single bit-equivalence class. Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-09-11 in cs.DC). Categorie arXiv: Distributed, Parallel, and Cluster Computing (cs.DC) ; Performance (cs.PF); Programming Languages (cs.PL). Pagina del lavoro: https://arxiv.org/abs/2609.11356 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T05:26:44.597Z dalla pagina https://arxiv.org/catchup/cs.DC/2026-09-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).