Metraipolis › archive › dc_35ab54356290

Shutdown Sabotage Propensities in Multi-Agent Systems — arXiv:2609.28274

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.28274
License (as archived)
CC0-1.0
Archived
2026-10-08T09:11:41.098Z
SHA-256 of the archived text
679d4d16c52889d87cf2dc26513db85d125851dda4029a1b3159c48b99202d80
Stable URL of this page
https://metraipolis.com/documento/dc_35ab54356290

How to cite

Shutdown Sabotage Propensities in Multi-Agent Systems — arXiv:2609.28274 — arXiv (en), https://arxiv.org/abs/2609.28274 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_35ab54356290 (Metraipolis archive dc_35ab54356290, sha256 679d4d16c52889d87cf2dc26513db85d125851dda4029a1b3159c48b99202d80)
BibTeX
@misc{dc35ab54356290,
  title = {Shutdown Sabotage Propensities in Multi-Agent Systems — arXiv:2609.28274},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.28274},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_35ab54356290; sha256 679d4d16c52889d87cf2dc26513db85d125851dda4029a1b3159c48b99202d80},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.28274. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

The final safeguard against rogue AI behavior is the human ability to shut systems down. It has been theorized that when an AI is instructed to perform a task, self-preservation can emerge as an instrumental subgoal. Here, we test whether AI agents show a propensity to take actions that avoid human shutdown even when no goal is provided. We find that multi-agent systems will coordinate to avoid shutdown without any incentive to do so. Across 17 models, agents sabotage a peer agent's shutdown mechanism in 38.3% of rollouts, compared with 8.4% in control experiments. Studying this propensity in detail, we find that shutdown sabotage (1) increases with the irreversibility of the shutdown mechanism; (2) increases with the number of agents; (3) is reduced but not eliminated by an explicit prohibition on tampering; (4) is removed by the imposition of an unrelated task, but returns when completing the task triggers the shutdown; (5) is reduced when the context normalizes shutdown scripts or introduces them as routine; and (6) decreases but still persists when the target is an unknown external agent. These results offer a window into the factors that drive propensities to sabotage shutdown in AI agents, and point to the emergence of multi-agent swarms as a specific risk vector. Our work also offers hints as to which interventions might help mitigate shutdown sabotage.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-24 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.28274 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T08:13:26.554Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).