Metraipolis › archive › dc_97a892e654bf
Training for Alignment — arXiv:2610.40684
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.40684
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T08:31:25.851Z
- SHA-256 of the archived text
33a85af023e845f78646957b80736cf64937c547a1a81507971dd2063843a9e8- Stable URL of this page
https://metraipolis.com/documento/dc_97a892e654bf
How to cite
Training for Alignment — arXiv:2610.40684 — arXiv (en), https://arxiv.org/abs/2610.40684 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_97a892e654bf (Metraipolis archive dc_97a892e654bf, sha256 33a85af023e845f78646957b80736cf64937c547a1a81507971dd2063843a9e8)
BibTeX
@misc{dc97a892e654bf,
title = {Training for Alignment — arXiv:2610.40684},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.40684},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_97a892e654bf; sha256 33a85af023e845f78646957b80736cf64937c547a1a81507971dd2063843a9e8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.40684. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
We propose a simple model of training an agent with an endogenous bias. A principal chooses in which states to train the agent and what actions to prescribe in these states. In response, the agent updates their bias and then acts optimally given their new bias in untrained states, while following their training in trained states. The agent chooses their new bias to minimize a weighted average of their loss under their initial and final biases. Training can thus generalize to untrained states via the agent's bias updating. We characterize optimal training, which involves \emph{over-correction} in trained states and \emph{generalization-aware} (non-greedy) selection of training states. If only a small share of states can be trained, greedy state selection is approximately optimal, but over-correction remains essential. If any bias function is possible, or if the principal evaluates training robustly over bias functions that admit the possibility of full alignment, the value of training generalization vanishes, and greedy training becomes optimal. With a more pessimistic objective, \emph{under-correction}, or even no training at all, may be optimal. The principal benefits from the agent's preferences being more flexible (\emph{corrigibility}) when the set of possible biases is known and contains full alignment, but otherwise corrigibility can be harmful. Applications include socialization in organizations and families, and AI post-training and recursive self-improvement.
Tema degli esploratori: economia dei servizi IA (elenco arXiv econ.TH, nuovi lavori del giorno).
Categorie arXiv: Theoretical Economics (econ.TH).
Pagina del lavoro: https://arxiv.org/abs/2610.40684 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T08:15:09.207Z dalla pagina https://arxiv.org/list/econ.TH/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).