Metraipolis › archive › dc_cbb881c86f2e

Stress-testing Alignment Midtraining — arXiv:2609.20412

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.20412
License (as archived)
CC0-1.0
Archived
2026-10-10T03:24:16.494Z
SHA-256 of the archived text
38c9846acd03f5dbb1a2741f72313af792ef438308bcd0b5b47493a96bd1f2c1
Stable URL of this page
https://metraipolis.com/documento/dc_cbb881c86f2e

How to cite

Stress-testing Alignment Midtraining — arXiv:2609.20412 — arXiv (en), https://arxiv.org/abs/2609.20412 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cbb881c86f2e (Metraipolis archive dc_cbb881c86f2e, sha256 38c9846acd03f5dbb1a2741f72313af792ef438308bcd0b5b47493a96bd1f2c1)
BibTeX
@misc{dccbb881c86f2e,
  title = {Stress-testing Alignment Midtraining — arXiv:2609.20412},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.20412},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cbb881c86f2e; sha256 38c9846acd03f5dbb1a2741f72313af792ef438308bcd0b5b47493a96bd1f2c1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.20412. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

When aligning frontier models through post-training techniques, it is not possible to directly demonstrate all of the behaviours we want a model to exhibit in all possible deployment environments; our model must generalise outside of the post-training distribution. One proposed solution is alignment midtraining (AMT), which continues pretraining on large volumes of alignment-relevant documents to encourage generalisation in later stages of training. Despite the prominence of AMT as an alignment approach, there is limited public evidence for its effectiveness. To resolve this, we identify several assumptions around midtraining and evaluate them across scale: up to 110 billion-parameter models and 1 billion midtraining tokens. For instance, we study a scenario where post-training data is ambiguous between two possible motivations. We find that midtraining can steer the model's motivation in simple versions of this setting. However, the presence of a tiny fraction of finetuning data which suggests a competing motivation erases the effects of AMT. We also study scenarios in which we want an AI to follow a number of rules, but only demonstrate a subset of them. We find that demonstrations must be present either in midtraining or post-training datasets for these rules to be robustly learned. Based on these and other findings, we do not believe that there is sufficient public evidence for us to confidently state that midtraining can address the core difficulties inherent in aligning powerful AI systems.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-18 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.20412 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-10T03:04:13.950Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).