Metraipolis › archive › dc_8fa36d7a8f08

Synthetic Persona Pretraining: Alignment from Token Zero — arXiv:2608.13482

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.13482
License (as archived)
CC0-1.0
Archived
2026-10-19T04:18:35.707Z
SHA-256 of the archived text
a089c35ba23646a2d9c2fc5d88cd260ee7c78ba03e60c2dcf63e8b21e27e6211
Stable URL of this page
https://metraipolis.com/documento/dc_8fa36d7a8f08

How to cite

Synthetic Persona Pretraining: Alignment from Token Zero — arXiv:2608.13482 — arXiv (en), https://arxiv.org/abs/2608.13482 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8fa36d7a8f08 (Metraipolis archive dc_8fa36d7a8f08, sha256 a089c35ba23646a2d9c2fc5d88cd260ee7c78ba03e60c2dcf63e8b21e27e6211)
BibTeX
@misc{dc8fa36d7a8f08,
  title = {Synthetic Persona Pretraining: Alignment from Token Zero — arXiv:2608.13482},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.13482},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8fa36d7a8f08; sha256 a089c35ba23646a2d9c2fc5d88cd260ee7c78ba03e60c2dcf63e8b21e27e6211},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.13482. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As language-model-based AI is increasingly deployed in autonomous settings, aligning its goals and values with those of humans becomes critical. Today, alignment, and the assistant identity itself, are typically introduced only after pretraining, once behavioral priors are already established. This can make values a thin overlay, rather than deeply rooted, and facilitate subsequent misalignment. Pursuing a different paradigm, we introduce Synthetic Persona Pretraining (SPP), which installs the desired assistant persona from token zero in pretraining. First, we annotate pretraining documents with value-aligned first-person reflections derived from a normative value constitution. Second, we pretrain via the standard cross-entropy loss on standard pretraining documents as well as their reflections, which installs the desired persona among a multitude of other personas. Finally, we post-train on user-assistant dialogue data, which binds this desired persona to the assistant identity, a process we call persona binding. By pretraining models up to 3B parameters on 500B tokens, we show that SPP improves constitution following and jailbreak robustness, and reduces the misalignment rate in out-of-distribution moral dilemmas, while preserving capabilities. Early intervention matters: compared with alignment from token zero, introducing SPP only at the end of pretraining yields weaker constitution adherence, does not shift value priorities, and leads to less aligned choices in dilemmas. This advantage depends on persona binding and, importantly, increases with pretraining budget. Overall, our results show that shaping values early is critical for alignment and establish pretraining-time persona interventions as an effective approach to do so.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-14 in cs.AI).
Categorie arXiv: Machine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2608.13482 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T04:13:06.778Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).