Metraipolis › archive › dc_e273984c5aff
Sparse RL Relaxation for LLM Reasoning — arXiv:2610.27220
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.27220
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-21T07:06:14.918Z
- SHA-256 of the archived text
5b5d8d462b560bd702304f9cc458c5a96d2910aae440ee15afbd123fee5c9bc8- Stable URL of this page
https://metraipolis.com/documento/dc_e273984c5aff
How to cite
Sparse RL Relaxation for LLM Reasoning — arXiv:2610.27220 — arXiv (en), https://arxiv.org/abs/2610.27220 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e273984c5aff (Metraipolis archive dc_e273984c5aff, sha256 5b5d8d462b560bd702304f9cc458c5a96d2910aae440ee15afbd123fee5c9bc8)
BibTeX
@misc{dce273984c5aff,
title = {Sparse RL Relaxation for LLM Reasoning — arXiv:2610.27220},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.27220},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e273984c5aff; sha256 5b5d8d462b560bd702304f9cc458c5a96d2910aae440ee15afbd123fee5c9bc8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27220. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Recent work has explored continuous representations in place of discrete tokens during Chain-of-Thought (CoT) reasoning, often referred to as soft CoT. This is motivated by the idea that a mixture of token embeddings can represent a superposition of multiple reasoning possibilities within a single step. However, it remains challenging to train large language models to realize this potential. Existing work has explored reinforcement learning (RL)-style training for soft CoT. However, these approaches treat soft tokens as actions in a Markov decision process rather than as differentiable intermediate computation steps. Thus, they do not propagate gradients across the soft tokens and miss out on the signal of how the choice of one soft token affects the next. In this paper, we propose a sparse relaxation method for RL (called SpaRLax), which leverages the differentiability of soft-CoT reasoning over time steps and makes training more stable and effective. We adopt an entmax sparsity method that adaptively determines the degree of softness, which truncates long-tail noise at confident steps while retaining multiple plausible candidates when uncertain. Across nine reasoning benchmarks, our method achieves the highest average performance compared with baseline methods. Further analyses validate the effective use of soft-token superposition in our approach, and also demonstrate the contribution of adaptive sparsity in our approach. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Artificial Intelligence (cs.AI). Pagina del lavoro: https://arxiv.org/abs/2610.27220 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-21T07:05:43.387Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).