Metraipolis › archive › dc_b430df5ceec4
MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning — arXiv:2610.32234
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.32234
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-26T20:11:53.962Z
- SHA-256 of the archived text
5999db2cb124f5eb5ab4031decc5f98dd561b8d89688fff1cb8e334cde7fda1f- Stable URL of this page
https://metraipolis.com/documento/dc_b430df5ceec4
How to cite
MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning — arXiv:2610.32234 — arXiv (en), https://arxiv.org/abs/2610.32234 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b430df5ceec4 (Metraipolis archive dc_b430df5ceec4, sha256 5999db2cb124f5eb5ab4031decc5f98dd561b8d89688fff1cb8e334cde7fda1f)
BibTeX
@misc{dcb430df5ceec4,
title = {MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning — arXiv:2610.32234},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.32234},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b430df5ceec4; sha256 5999db2cb124f5eb5ab4031decc5f98dd561b8d89688fff1cb8e334cde7fda1f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32234. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Scientific, engineering, and mathematical reasoning tasks can admit multiple valid solutions, while exhaustive annotation of the full solution set is often prohibitively costly. Test-time reinforcement learning (TTRL) applies reinforcement learning to unlabeled test instances, offering a label-free approach to multiple-solution reasoning. In this setting, generation frequency serves two opposing objectives: correctness estimation favors frequent outcomes, whereas outcome-level diversity preservation favors less frequent outcomes. We term this tension the \textbf{Frequency Conflict}. To resolve this conflict, we propose MS-TTRL, which obtains frequency-independent correctness estimates through Outcome-Level Self-Verification while reserving generation frequency solely for diversity preservation. Three-State Outcome Cache and the Candidate State support verification reuse across repeated observations and confidence-aware handling of ambiguous outcomes, while Confidence-Gated Inverse Probability Scaling selectively strengthens the learning signals of rare, reliably verified outcomes. Experiments across two benchmarks, four multiple-solution tasks, and three backbone models, each evaluated with five random seeds, show that MS-TTRL improves Recovery Rate by 19.13 percentage points on average over the best-performing label-free baseline and approaches the performance of the supervised IPS-GRPO.
Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.LG, nuovi lavori del giorno).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.32234 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:49:03.389Z dalla pagina https://arxiv.org/list/cs.LG/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).