Metraipolis › archive › dc_35a58b757e89

Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space — arXiv:2609.33271

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33271
License (as archived)
CC0-1.0
Archived
2026-10-07T20:56:34.994Z
SHA-256 of the archived text
ce75017ba4b7ce7efdafaf8cfa80781327a19b5ce909064d3cdb1171ac87b0d2
Stable URL of this page
https://metraipolis.com/documento/dc_35a58b757e89

How to cite

Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space — arXiv:2609.33271 — arXiv (en), https://arxiv.org/abs/2609.33271 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_35a58b757e89 (Metraipolis archive dc_35a58b757e89, sha256 ce75017ba4b7ce7efdafaf8cfa80781327a19b5ce909064d3cdb1171ac87b0d2)
BibTeX
@misc{dc35a58b757e89,
  title = {Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space — arXiv:2609.33271},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33271},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_35a58b757e89; sha256 ce75017ba4b7ce7efdafaf8cfa80781327a19b5ce909064d3cdb1171ac87b0d2},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33271. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Reasoning problems often admit multiple valid ways to proceed. Continuous reasoning promises to move computation beyond language tokens into a more compact latent space, but representing several plausible ways to think next remains difficult. We introduce Autoregressive Thought Flow (ATF), which models the next continuous thought as a multimodal distribution. A causal autoregressive model performs the reasoning computation, while a lightweight diffusion head generates a plausible next thought from the resulting condition. The sampled thought is fed back into the model, allowing continuous reasoning to unfold for a variable number of steps while preserving the pretrained backbone. Across mathematical reasoning tasks, ATF improves accuracy with compact latent traces and benefits from reinforcement learning and additional test-time thinking. Multi-sample evaluation shows broader solution coverage, indicating that its multimodal predictions capture useful diversity among reasoning paths. Our results suggest that continuous reasoning is more effective when multiple possible next thoughts remain available rather than being collapsed into a single prediction.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.33271 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.191Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).