Metraipolis › archive › dc_6dd44ecffca3

SeOPD: Self-Evolving LLMs via Online Policy Distillation from Self-Generated Chain-of-Thought — arXiv:2609.33181

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.33181
License (as archived): CC0-1.0
Archived: 2026-10-07T20:56:30.452Z
SHA-256 of the archived text: 6f42db87e913b3610354f18e7d5ffee18989a84111886019d46daccd68070756
Stable URL of this page: https://metraipolis.com/documento/dc_6dd44ecffca3

How to cite

SeOPD: Self-Evolving LLMs via Online Policy Distillation from Self-Generated Chain-of-Thought — arXiv:2609.33181 — arXiv (en), https://arxiv.org/abs/2609.33181 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_6dd44ecffca3 (Metraipolis archive dc_6dd44ecffca3, sha256 6f42db87e913b3610354f18e7d5ffee18989a84111886019d46daccd68070756)

BibTeX

@misc{dc6dd44ecffca3,
  title = {SeOPD: Self-Evolving LLMs via Online Policy Distillation from Self-Generated Chain-of-Thought — arXiv:2609.33181},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33181},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_6dd44ecffca3; sha256 6f42db87e913b3610354f18e7d5ffee18989a84111886019d46daccd68070756},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33181. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent advances in online policy self-distillation (OPSD) have demonstrated that large language models (LLMs) can improve their capabilities by leveraging external privileged information (PI), such as manual annotations or feedback from external environments. However, obtaining accurate annotations and constructing sophisticated environments often require substantial human effort and computation, limiting the scalability of OPSD. While a few recent studies have explored self-improvement without external PI, the resulting gains remain limited. In this work, we explore whether LLMs can achieve comparable self-improvement without external PI. Our key observation is that a single LLM can support multiple reasoning modes, such as deep-thinking and non-thinking modes, with deep thinking generating additional information during reasoning. Based on this observation, we propose Self-Evolving Online Policy Distillation (SeOPD), which enables LLMs to distill and internalize information generated by their own chain of thought (CoT). Specifically, it (1) generates CoT with the deep-thinking mode, (2) produces responses with the non-thinking mode, and (3) uses the generated CoT as PI to provide token-level supervision for the non-thinking response, allowing new information inferred during reasoning to guide the non-thinking mode and be internalized into the shared model parameters, thereby improving both non-thinking and deep-thinking capabilities. Extensive experiments across LLMs and tasks demonstrate the effectiveness of SeOPD.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.33181 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:29:01.170Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).