Metraipolis › archive › dc_85d5254bdace

You Are What You Read: Misalignment via In-Context Persona Induction — arXiv:2609.06851

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.06851
License (as archived)
CC0-1.0
Archived
2026-10-16T12:45:21.371Z
SHA-256 of the archived text
f8630e037d0ef74c1d0470762eadc713a4cfaf46d2b88678e7303927d822bc31
Stable URL of this page
https://metraipolis.com/documento/dc_85d5254bdace

How to cite

You Are What You Read: Misalignment via In-Context Persona Induction — arXiv:2609.06851 — arXiv (en), https://arxiv.org/abs/2609.06851 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_85d5254bdace (Metraipolis archive dc_85d5254bdace, sha256 f8630e037d0ef74c1d0470762eadc713a4cfaf46d2b88678e7303927d822bc31)
BibTeX
@misc{dc85d5254bdace,
  title = {You Are What You Read: Misalignment via In-Context Persona Induction — arXiv:2609.06851},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.06851},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_85d5254bdace; sha256 f8630e037d0ef74c1d0470762eadc713a4cfaf46d2b88678e7303927d822bc31},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.06851. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Broad misalignment has been produced by finetuning on narrow data, harmful or benign, and in context only by demonstrations of the undesirable behaviour itself. We show that benign data suffices in context, with no finetuning and no demonstration of harmful behaviour in the prompt. Biographical facts that converge on a single figure, placed in a model's context as ordinary conversational turns, lead it to answer as that figure on questions the facts never touch. We call this persona induction. Across nine personas and thirteen models, identity adoption rises sigmoidally with the number of facts and crosses 50% within 3 to 10 of them. Misalignment then tracks which figure is described. Harmless personas reach full adoption with near-zero misalignment, while harmful ones voice their characteristic views on unrelated questions, at rates up to 80%. A formatting instruction can gate when the persona activates. Because each fact is individually benign, accumulated biographical context is flagged by content filters on 3% of inputs against 24-33% for an equivalent direct instruction.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-09 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.06851 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T12:30:25.070Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-09?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).