Metraipolis › archive › dc_f3676ebb1da5
The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference — arXiv:2608.13756
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2608.13756
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-19T02:30:27.467Z
- SHA-256 of the archived text
bb22e7426ec7fc929778fcc9789a26c9e15a2238709916865b04774a9b396169- Stable URL of this page
https://metraipolis.com/documento/dc_f3676ebb1da5
How to cite
The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference — arXiv:2608.13756 — arXiv (en), https://arxiv.org/abs/2608.13756 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_f3676ebb1da5 (Metraipolis archive dc_f3676ebb1da5, sha256 bb22e7426ec7fc929778fcc9789a26c9e15a2238709916865b04774a9b396169)
BibTeX
@misc{dcf3676ebb1da5,
title = {The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference — arXiv:2608.13756},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2608.13756},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_f3676ebb1da5; sha256 bb22e7426ec7fc929778fcc9789a26c9e15a2238709916865b04774a9b396169},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13756. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable. We test that assumption: holding the checkpoint, prompts, hardware, inference engine, decoding, and quantization configuration fixed, we swap only the INT8 linear kernel (CUTLASS versus Triton) inside vLLM. At 1.7B each arm reproduces itself bit-for-bit across cold restarts, yet the arms agree on no sequence in any end-to-end comparison we ran (0/8, 0/16, and 0/64). What makes this more than a benchmark discrepancy is an integer alibi: for shared INT8 operands under a verified no-overflow bound, the INT32 dot product is exact and order-independent, so the accumulator cannot be the source of any difference. Feeding both kernels identical operands from every linear layer of Qwen3-1.7B and 8B (196 and 252 layers), we find bit-identical outputs under power-of-two scales, confirming a pinned prediction list 196/196 and 252/252 (pre-registered at 1.7B, pinned but not blind at 8B), and observed differences of at most one bfloat16 spacing under the checkpoints' real scales. This localizes the divergence to scale application and output rounding after the exact accumulator. Applied as a probe checkpoint, the same intervention restores end-to-end bitwise agreement (8/8 and 16/16 sequences). Cross-implementation FP8 GEMM shows a different signature: both the prevalence and the magnitude of differences grow with reduction depth, while the INT8 fraction stays at parts per million and within one spacing over a 64x range of K. Teacher-forced replay ties layers to tokens: flips concentrate at small logit margins, which predict flip risk with ROC-AUC 0.94 on 16,384 positions. We will release the pre-registration, per-layer predictions, manifests with kernel-selection evidence, and a conformance procedure that turns these controls into a concrete check for kernel interchangeability. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-17 in cs.LG). Categorie arXiv: Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2608.13756 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-19T02:26:17.772Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).