Metraipolis › archive › dc_af4f9bc112a4

AutoEMatic: Runtime-Adaptive Mixed-Precision LLM Inference via Automatic Exponent-Mantissa Refinement — arXiv:2611.10165

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.10165
License (as archived)
CC0-1.0
Archived
2026-11-10T18:40:46.433Z
SHA-256 of the archived text
883faefde48e2fe8c84036cc0cd7d7602eb569ed19a0b52b7596766efb4dc5b7
Stable URL of this page
https://metraipolis.com/documento/dc_af4f9bc112a4

How to cite

AutoEMatic: Runtime-Adaptive Mixed-Precision LLM Inference via Automatic Exponent-Mantissa Refinement — arXiv:2611.10165 — arXiv (en), https://arxiv.org/abs/2611.10165 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_af4f9bc112a4 (Metraipolis archive dc_af4f9bc112a4, sha256 883faefde48e2fe8c84036cc0cd7d7602eb569ed19a0b52b7596766efb4dc5b7)
BibTeX
@misc{dcaf4f9bc112a4,
  title = {AutoEMatic: Runtime-Adaptive Mixed-Precision LLM Inference via Automatic Exponent-Mantissa Refinement — arXiv:2611.10165},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.10165},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_af4f9bc112a4; sha256 883faefde48e2fe8c84036cc0cd7d7602eb569ed19a0b52b7596766efb4dc5b7},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.10165. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Existing LLM post-training quantization techniques determine precision before inference begins, yet the precision a model requires varies dynamically across prompts, layers, and inference phases. We present AutoEMatic, a full-stack co-design that adapts the precision of weights, activations, and the KV cache at runtime. A lightweight policy selects each layer's precision at every prefill and decode step from two runtime signals: the entropy of the next-token distribution and the magnitude of each layer's update to its hidden states. This runtime adaptation is made practical by AutoFP, a multi-scale floating-point format that derives 2-, 4-, and 6-bit precisions from a single NVFP4 base format. We realize AutoEMatic on two targets: GPU kernels that execute AutoFP on NVFP4 Tensor Cores, and an accelerator whose vector MAC datapath executes all AutoFP formats natively. Across reasoning and agentic benchmarks, AutoEMatic improves accuracy by up to 21.7% over NVFP4 and matches group-scaled FP6 accuracy on most benchmarks, using 4.9 bits per weight versus 4.5 for NVFP4. On GB300 GPUs, AutoEMatic retains approximately 90-98% of NVFP4 decode throughput across the evaluated configurations. Furthermore, the AutoEMatic accelerator reduces latency by 1.6-2.3x and energy by 7-23% compared to an accelerator with NVFP4-only datapath at iso-accuracy.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2611.10165 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:37.479Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).