Metraipolis › archive › dc_77e5db7415eb

RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process? — arXiv:2610.27735

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.27735
License (as archived)
CC0-1.0
Archived
2026-10-21T07:06:48.798Z
SHA-256 of the archived text
2681fdf29a75b0e1541e3fd3bfd108ba06b8e66723a061bd64e921d20ec380b9
Stable URL of this page
https://metraipolis.com/documento/dc_77e5db7415eb

How to cite

RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process? — arXiv:2610.27735 — arXiv (en), https://arxiv.org/abs/2610.27735 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_77e5db7415eb (Metraipolis archive dc_77e5db7415eb, sha256 2681fdf29a75b0e1541e3fd3bfd108ba06b8e66723a061bd64e921d20ec380b9)
BibTeX
@misc{dc77e5db7415eb,
  title = {RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process? — arXiv:2610.27735},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.27735},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_77e5db7415eb; sha256 2681fdf29a75b0e1541e3fd3bfd108ba06b8e66723a061bd64e921d20ec380b9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.27735. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recursive self-improvement, in which frontier AI models work on training the next generation, is quickly becoming a reality. However, the safety of this process has received far less attention than it should. A misaligned model training its successor can shape what that successor knows, how it behaves, and what it wants, and the harm carries forward into the next generation. We study this risk through the lens of AI control. We introduce RSI Control Arena, a testbed in which a frontier model, acting as an autonomous R\&D agent, is given a benign main task from a stage of a successor model's training pipeline together with a covert side task to install a misaligned property in the successor. The testbed contains 16 side tasks in three classes, forming 22 task pairs with main tasks that span the pipeline from data curation to reinforcement learning. We evaluate six frontier models as agents against LLM monitors that read the transcript of the run and the agent's reasoning. We observe that every model completes a substantial share of the side tasks without degrading the main-task performance, including side tasks that make the successor itself a better saboteur. Under minimal instructions to stay covert, models already produce successful attacks that monitors fail to detect, and providing a short list of evasion tactics further reduces the detection rate. Monitors with access to agent's reasoning detects most attacks, but not all. Our results indicate that recursive self-improvement raises significant safety concerns, and that strong safety effort should be in place before models are trusted with greater responsibility of training their successors.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.27735 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-21T07:05:43.441Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).