Metraipolis › archive › dc_4f5e53bdbdee
Compositional Reasoning in Language Models under Reinforcement Learning Post-Training — arXiv:2609.19465
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.19465
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-10T03:19:53.000Z
- SHA-256 of the archived text
aaf7665e3aa8eb6c410baa7de9d2bde6387ef103500fda3961c2b95277ba099e- Stable URL of this page
https://metraipolis.com/documento/dc_4f5e53bdbdee
How to cite
Compositional Reasoning in Language Models under Reinforcement Learning Post-Training — arXiv:2609.19465 — arXiv (en), https://arxiv.org/abs/2609.19465 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4f5e53bdbdee (Metraipolis archive dc_4f5e53bdbdee, sha256 aaf7665e3aa8eb6c410baa7de9d2bde6387ef103500fda3961c2b95277ba099e)
BibTeX
@misc{dc4f5e53bdbdee,
title = {Compositional Reasoning in Language Models under Reinforcement Learning Post-Training — arXiv:2609.19465},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.19465},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4f5e53bdbdee; sha256 aaf7665e3aa8eb6c410baa7de9d2bde6387ef103500fda3961c2b95277ba099e},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19465. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Compositional reasoning is critical for real-world problem solving: since training data is necessarily limited, models must generalize by composing learned skills in new ways. While post-training methods such as reinforcement learning (RL) have substantially improved the reasoning abilities of language models (LMs), their effects on compositional reasoning remain less well understood. We propose a dependency-graph framework to formalize compositional reasoning, yielding three levels of compositionality with increasing complexity. Empirically, we instantiate this framework with data-structure tasks, which provide deterministic reward computation and clear compositional structure. We find a consistent decomposed-to-composed asymmetry: decomposed-skill training does not reliably transfer to composed tasks, whereas composed-task training transfers more readily back to decomposed tasks. We provide theoretical explanation for this asymmetry, and further evaluate compositional generalization under length extrapolation, structural distribution shift, and transfer to tasks requiring unseen skills. Finally, we present a pilot study on real-world tool-calling benchmarks, showing preliminary evidence that the decomposed-to-composed asymmetry can extend to practical settings. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.AI). Categorie arXiv: Artificial Intelligence (cs.AI) ; Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2609.19465 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-10T03:04:13.602Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).