Metraipolis › archive › dc_37e4fdb03052
Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe — arXiv:2610.22019
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.22019
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-16T09:30:53.993Z
- SHA-256 of the archived text
b7f967fdfac1877ec75868f933db519122156f58d113cff4c3a62bd119aedc9d- Stable URL of this page
https://metraipolis.com/documento/dc_37e4fdb03052
How to cite
Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe — arXiv:2610.22019 — arXiv (en), https://arxiv.org/abs/2610.22019 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_37e4fdb03052 (Metraipolis archive dc_37e4fdb03052, sha256 b7f967fdfac1877ec75868f933db519122156f58d113cff4c3a62bd119aedc9d)
BibTeX
@misc{dc37e4fdb03052,
title = {Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe — arXiv:2610.22019},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.22019},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_37e4fdb03052; sha256 b7f967fdfac1877ec75868f933db519122156f58d113cff4c3a62bd119aedc9d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.22019. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Frontier models, general-purpose and domain-specific alike, now include a midtraining stage between pretraining and post-training, but little has been published about how this stage is built. We present an open midtraining recipe that primes base models for agentic post-training, and we show its effectiveness on two Nemotron models of different sizes. Pretraining typically runs at short context and sees almost no agent interactions, while SFT data is mostly synthetic, and very long SFT stages can hurt the model's later RL trainability. Our midtraining stage sits between the two. It trains at 256K context for hundreds of billions of tokens on a diverse mixture of natural software-engineering data like packed repositories, pull requests, commit histories, and build logs together with a small share of agentic traces. On Nemotron 3.5 Lightning, 500B midtraining tokens improve SWE-bench Verified from 60.7 to 64.5, SWE-bench Multilingual from 55.8 to 62.8, and SWE-bench Pro from 32.0 to 35.5 after identical full SFT. On the larger Nemotron 3.5 Super, 280B midtraining tokens improve DeepSWE, a harder benchmark of long-horizon tasks, from 11.7 to 15.7 and SWE-bench Verified from 72.9 to 74.3 after identical full SFT. Midtraining also improves robustness across agent harnesses on both models. We report the ablations behind the recipe and an evaluation protocol built on short fixed SFT runs, since base-model proxy metrics did not correlate reliably with post-training results. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.22019 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-16T08:44:18.018Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).