Metraipolis › archive › dc_d3d270fd8fd2

Principled Thoughts for Latent Recursive LLM Systems — arXiv:2609.36159

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.36159
License (as archived)
CC0-1.0
Archived
2026-10-07T16:22:20.151Z
SHA-256 of the archived text
b8894b6c632724fd068c95d015158805d9f7f9bf45888f91df3d2388d948f37d
Stable URL of this page
https://metraipolis.com/documento/dc_d3d270fd8fd2

How to cite

Principled Thoughts for Latent Recursive LLM Systems — arXiv:2609.36159 — arXiv (en), https://arxiv.org/abs/2609.36159 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d3d270fd8fd2 (Metraipolis archive dc_d3d270fd8fd2, sha256 b8894b6c632724fd068c95d015158805d9f7f9bf45888f91df3d2388d948f37d)
BibTeX
@misc{dcd3d270fd8fd2,
  title = {Principled Thoughts for Latent Recursive LLM Systems — arXiv:2609.36159},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.36159},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d3d270fd8fd2; sha256 b8894b6c632724fd068c95d015158805d9f7f9bf45888f91df3d2388d948f37d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36159. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large language models can reason in continuous space instead of decoded text, by recurring on their own hidden states or by passing those states between agents, while training supervises only the Cross-Entropy (CE) of the final decoded answer and does not constrain the thought. Theoretical and empirical analyses establish and confirm four failures of CE-only training that lead to a lower probability of the correct answer such as collapsing thoughts across distinct questions and retaining irrelevant information. We introduce REST (REpresentation-Supervised Thoughts), a training objective that turns four properties of a valid thought representation (causality, minimality, separability, and stability) into differentiable losses added to CE. We instantiate it in latent single-agent and multi-agent systems, without architectural changes or added parameters at inference. Across 7 benchmarks spanning mathematics, science, medicine, and code generation, with the same training data, compute, and latent budget, REST increases accuracy over CE-only training across agent settings and model sizes by up to 7.5 percentage points and convergence on a final answer by 30\%. Furthermore, REST thoughts encode more of what is required to achieve the correct answer, and decoding them better recovers the intended output of the agent, which makes latent communication easier to interpret. Project Website: this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-30 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.36159 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T15:54:53.472Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).