Metraipolis › archive › dc_b6722af67972
Where Rounding Happens Matters: Rounding-Location Alignment Stabilizes LLM Reinforcement Learning — arXiv:2610.32951
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.32951
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T07:59:20.316Z
- SHA-256 of the archived text
ffcfc1cc9a4ce3eedca399e4ef8f4d47230856da55379b1d754eb54d395027f2- Stable URL of this page
https://metraipolis.com/documento/dc_b6722af67972
How to cite
Where Rounding Happens Matters: Rounding-Location Alignment Stabilizes LLM Reinforcement Learning — arXiv:2610.32951 — arXiv (en), https://arxiv.org/abs/2610.32951 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b6722af67972 (Metraipolis archive dc_b6722af67972, sha256 ffcfc1cc9a4ce3eedca399e4ef8f4d47230856da55379b1d754eb54d395027f2)
BibTeX
@misc{dcb6722af67972,
title = {Where Rounding Happens Matters: Rounding-Location Alignment Stabilizes LLM Reinforcement Learning — arXiv:2610.32951},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.32951},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b6722af67972; sha256 ffcfc1cc9a4ce3eedca399e4ef8f4d47230856da55379b1d754eb54d395027f2},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32951. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Reinforcement learning for large language models often uses separate systems for inference and training. Even with identical weights and BF16 tensors, implementation differences can make an intended on-policy update off-policy and destabilize training. Existing remedies align the entire inference and training stacks, change global precision, or modify the learning algorithm. We ask whether aligning selected parts of the execution stack can stabilize BF16 RL while other implementation differences remain. We align training-side rounding locations with inference in three regions---RoPE, SwiGLU, and residual RMSNorm---by retaining selected intermediates in FP32 until their BF16 outputs. Across four Qwen-family models, aligning all three regions prevents the collapse observed under the default BF16 baseline; RoPE-path alignment alone also suffices in the studied runs, despite remaining numerical discrepancies. The interventions keep the inference engine, matrix-multiplication and attention kernels, and learning objective unchanged. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.32951 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:55:30.777Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).