Metraipolis › archive › dc_2cf73fc3b836

Learn Your Own Thoughts: Abstract Token Curriculum — arXiv:2609.19717

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.19717
License (as archived)
CC0-1.0
Archived
2026-10-10T03:23:32.617Z
SHA-256 of the archived text
2fb86b1a7ac52aeecc7e24381b4f2539bae8e4ad19d1f9b944f02c87fbd8da1e
Stable URL of this page
https://metraipolis.com/documento/dc_2cf73fc3b836

How to cite

Learn Your Own Thoughts: Abstract Token Curriculum — arXiv:2609.19717 — arXiv (en), https://arxiv.org/abs/2609.19717 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2cf73fc3b836 (Metraipolis archive dc_2cf73fc3b836, sha256 2fb86b1a7ac52aeecc7e24381b4f2539bae8e4ad19d1f9b944f02c87fbd8da1e)
BibTeX
@misc{dc2cf73fc3b836,
  title = {Learn Your Own Thoughts: Abstract Token Curriculum — arXiv:2609.19717},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.19717},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2cf73fc3b836; sha256 2fb86b1a7ac52aeecc7e24381b4f2539bae8e4ad19d1f9b944f02c87fbd8da1e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.19717. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Large Language Models (LLMs) have achieved remarkable reasoning capabilities by utilizing chain-of-thought (CoT) as a scratchpad for intermediate stages of thinking. However, CoT techniques require explicit supervision on thinking tokens, which requires rich, task-specific data. In this work, we propose Abstract Token Curriculum (ATC), a novel curriculum learning framework that elicits effective continuous intermediate representations without direct supervision or manual scratchpad design. ATC gradually increases problem complexity through a sequence of distributions, training the model to develop internal abstract ``thoughts'' in the continuous representation space. This paper provides both theoretical and experimental evidence for the benefits of ATC and its advantages over previous methods for training continuous thoughts. Theoretically, we show that for learning parity functions with single-layer softmax attention using ATC, attention naturally focuses on the CoT tokens in the context that provide the ``easiest path'' to predicting the next token. Experimentally, we show ATC's effectiveness on graph reachability and arithmetic learning tasks.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (stat.ML).
Pagina del lavoro: https://arxiv.org/abs/2609.19717 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:04:13.848Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).