Metraipolis › archive › dc_99ed2bd4f104

Think Wider: Mitigating Latent Rank Collapse in Implicit Chain-of-Thought Reasoning — arXiv:2609.07406

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.07406
License (as archived)
CC0-1.0
Archived
2026-10-16T13:57:45.856Z
SHA-256 of the archived text
f685caa38828aacb80f4604c74865978983b6dcbcfe2533a2ddd069e8a99963c
Stable URL of this page
https://metraipolis.com/documento/dc_99ed2bd4f104

How to cite

Think Wider: Mitigating Latent Rank Collapse in Implicit Chain-of-Thought Reasoning — arXiv:2609.07406 — arXiv (en), https://arxiv.org/abs/2609.07406 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_99ed2bd4f104 (Metraipolis archive dc_99ed2bd4f104, sha256 f685caa38828aacb80f4604c74865978983b6dcbcfe2533a2ddd069e8a99963c)
BibTeX
@misc{dc99ed2bd4f104,
  title = {Think Wider: Mitigating Latent Rank Collapse in Implicit Chain-of-Thought Reasoning — arXiv:2609.07406},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.07406},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_99ed2bd4f104; sha256 f685caa38828aacb80f4604c74865978983b6dcbcfe2533a2ddd069e8a99963c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.07406. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Chain-of-thought (CoT) reasoning improves the reasoning ability of large language models by introducing intermediate computation, but explicit rationales increase decoding length, latency, and context cost. Implicit CoT offers a more efficient alternative by moving intermediate reasoning into continuous latent states. However, latent reasoning can be unstable: successive latent states may become overly similar and collapse toward a shared dominant direction, reducing the diversity of the reasoning trajectory. In this work, we identify $\textit{latent rank collapse}$ and propose $\textbf{WIDER}$, a lightweight spectral regularizer for implicit CoT. During training, WIDER estimates the shared direction of each latent trajectory and penalizes projections onto this direction, encouraging latent states to span a broader representational subspace. The method is plug-and-play and leaves the backbone model, latent schedule, and inference-time decoding procedure unchanged. We further formulate this collapse as a geometric bottleneck in implicit reasoning, casting its mitigation as a training-time regularization problem rather than an inference-time decoding change. Extensive experiments show that WIDER improves matched implicit CoT baselines, while mechanistic analyses reveal higher effective rank, lower dominant-direction energy, and reduced redundancy among latent steps. These results highlight latent subspace utilization as an important factor for efficient continuous reasoning, providing a geometric perspective for analyzing and improving implicit CoT. Code is available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2609.07406 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T13:39:42.951Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).