Metraipolis › archive › dc_63d9147bf3e5

Sep-Latent: Separator Tokens as Natural Compression Anchors for Implicit Reasoning — arXiv:2610.22924

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.22924
License (as archived)
CC0-1.0
Archived
2026-10-19T08:42:20.217Z
SHA-256 of the archived text
49d36f8fdabb3db85526c68777e0b1bd27a37c2ae382892b67e5213280481227
Stable URL of this page
https://metraipolis.com/documento/dc_63d9147bf3e5

How to cite

Sep-Latent: Separator Tokens as Natural Compression Anchors for Implicit Reasoning — arXiv:2610.22924 — arXiv (en), https://arxiv.org/abs/2610.22924 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_63d9147bf3e5 (Metraipolis archive dc_63d9147bf3e5, sha256 49d36f8fdabb3db85526c68777e0b1bd27a37c2ae382892b67e5213280481227)
BibTeX
@misc{dc63d9147bf3e5,
  title = {Sep-Latent: Separator Tokens as Natural Compression Anchors for Implicit Reasoning — arXiv:2610.22924},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.22924},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_63d9147bf3e5; sha256 49d36f8fdabb3db85526c68777e0b1bd27a37c2ae382892b67e5213280481227},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22924. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) have demonstrated remarkable success in complex reasoning through explicit Chain-of-Thought (CoT), but the associated computational overhead and token verbosity remain significant barriers to efficiency. While implicit reasoning in continuous latent spaces offers a promising alternative, existing methods like Coconut and CODI often rely on fixed-length latent sequences and suffer from representational collapse. In this paper, we propose Sep-Latent, an adaptive implicit reasoning framework that utilizes separator tokens as structural semantic anchors. By introducing an Attention-Decay mechanism, we force the model to aggregate and compress reasoning logic into these anchors during training. We then align explicit and implicit pathways through multi-level distillation across all layers to ensure logical consistency. Crucially, Sep-Latent enables dynamic reasoning depth by adaptively terminating the latent thinking phase based on query complexity. Experimental results on the GSM8k-Aug benchmark show that Sep-Latent achieves an accuracy of 56.8\% with an average of only 5.4 tokens, significantly outperforming prior fixed-length implicit models in efficiency while narrowing the performance gap with explicit CoT. Further analysis via PCA and symbolic variant testing confirms that our method ensures semantic grounding and exhibits superior robustness to perturbations.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.22924 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:39:10.046Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).