Metraipolis › archive › dc_bf6cf520c980

MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training — arXiv:2608.10823

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.10823
License (as archived)
CC0-1.0
Archived
2026-10-19T06:19:27.235Z
SHA-256 of the archived text
f291470b0fa96400afd3bafa7bf0d3abf0940778fed84b03feee1375f3f6d092
Stable URL of this page
https://metraipolis.com/documento/dc_bf6cf520c980

How to cite

MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training — arXiv:2608.10823 — arXiv (en), https://arxiv.org/abs/2608.10823 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bf6cf520c980 (Metraipolis archive dc_bf6cf520c980, sha256 f291470b0fa96400afd3bafa7bf0d3abf0940778fed84b03feee1375f3f6d092)
BibTeX
@misc{dcbf6cf520c980,
  title = {MoE Proxy Models for Low-Cost Failure Reproduction and Diagnosis in LLM RL Post-Training — arXiv:2608.10823},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.10823},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bf6cf520c980; sha256 f291470b0fa96400afd3bafa7bf0d3abf0940778fed84b03feee1375f3f6d092},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.10823. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reinforcement learning (RL) post-training of large language models (LLMs) is computationally intensive and involves complex system pipelines with substantial debugging overhead. In practice, factors such as framework adaptation, numerical precision, and operator implementation can cause failures, including gradient overflow and loss divergence. Reproducing such failures directly on large models requires considerable time and computational resources. This paper systematically analyzes failures encountered during large-scale RL training on the Huawei Ascend platform, summarizes representative failure types, and identifies three model-side factors relevant to fault reproduction. Based on these factors, we propose a proxy-model construction method for low-cost fault investigation and auxiliary diagnosis. It employs structure-preserving, clustering-based expert pruning to select representative experts while retaining the model's backbone architecture, routing mechanism, and basic task capabilities. Our experimental results show that the proxy models reduce accelerator requirements by 50%-87.5% and achieve up to a 33.3x reduction in per-step NPU-hour cost, while preserving major training dynamics and reproducing fault responses consistent with the original models. Overall, the proxy models can serve as low-cost surrogates for fault reproduction, targeted validation, and auxiliary diagnosis in RL post-training.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-12 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2608.10823 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T06:11:45.094Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-08-12?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).