Metraipolis › archive › dc_a3525940e050

On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models — arXiv:2610.32770

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32770
License (as archived)
CC0-1.0
Archived
2026-10-26T19:52:38.073Z
SHA-256 of the archived text
844e45dce479f6d0244c28f642a3b3f48d7703803fe469b96138d2d7c253ad4c
Stable URL of this page
https://metraipolis.com/documento/dc_a3525940e050

How to cite

On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models — arXiv:2610.32770 — arXiv (en), https://arxiv.org/abs/2610.32770 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a3525940e050 (Metraipolis archive dc_a3525940e050, sha256 844e45dce479f6d0244c28f642a3b3f48d7703803fe469b96138d2d7c253ad4c)
BibTeX
@misc{dca3525940e050,
  title = {On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models — arXiv:2610.32770},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32770},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a3525940e050; sha256 844e45dce479f6d0244c28f642a3b3f48d7703803fe469b96138d2d7c253ad4c},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32770. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

On-policy self-distillation enables post-training foundation models without relying on stronger external teachers. However, existing approaches rely on ground-truth question-answer pairs, human-curated problem sets, or other supervised signals. In this work, we reformulate on-policy self-distillation as a self-training framework, driven by self-generated question-answer pairs and privileged reasoning demonstrations. We push supervision to the limit: guided solely by coarse, high-level topic descriptions, the model synthesizes its own curriculum of queries and privileged reasoning traces without access to any human-curated prompts or answers. We find that self-distillation on fully synthetic privileged information enables a model to improve across multiple domains while retaining its capabilities on domains it is not self-trained on. We demonstrate the generality of this framework across text and multimodal domains, achieving consistent self-improvement on standard language benchmarks and Visual Question Answering. Our findings demonstrate that foundation models possess rich latent capabilities that can be amplified without external task supervision or curated problem distributions.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG).
Pagina del lavoro: https://arxiv.org/abs/2610.32770 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:45:33.433Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).