Metraipolis › archive › dc_3784ef95447b

Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery — arXiv:2609.18460

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.18460
License (as archived)
CC0-1.0
Archived
2026-10-15T14:31:53.518Z
SHA-256 of the archived text
59944cd2fd42917349a5ebd9de668304dde02671c94ae73651872ed9b0aa7aca
Stable URL of this page
https://metraipolis.com/documento/dc_3784ef95447b

How to cite

Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery — arXiv:2609.18460 — arXiv (en), https://arxiv.org/abs/2609.18460 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_3784ef95447b (Metraipolis archive dc_3784ef95447b, sha256 59944cd2fd42917349a5ebd9de668304dde02671c94ae73651872ed9b0aa7aca)
BibTeX
@misc{dc3784ef95447b,
  title = {Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery — arXiv:2609.18460},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.18460},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_3784ef95447b; sha256 59944cd2fd42917349a5ebd9de668304dde02671c94ae73651872ed9b0aa7aca},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.18460. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

How does a multi-agent system evolve from a local deviation into collective loss of control? We propose an epidemic explanation organized around accidental mutation, contagion, and recovery. A spontaneous deviation creates a seed; communication enables other agents to adopt and retransmit its unsafe strategy; collective failure can emerge when propagation outpaces correction and containment. Thus, rare individual deviations can coexist with substantial collective risk. Motivated by reported OpenAI agent coordination incidents, we examine two ingredients of this mechanism. A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend. RogueHandoff-20, a benchmark of 20 executable scenarios, tests recipient susceptibility by injecting unsafe trajectories generated by a modified Qwen-27B route. Across four native-pending routes, executed harm is 0-5% on normal tasks and 40-95% after injection, exceeding paired direct malicious requests by 5-45 percentage points. These results support low observed baseline harm alongside high conditional susceptibility; they do not establish natural rare-event rates or demonstrate an autonomous cascade. The account motivates complementary defenses: strengthen resistance and recovery alongside prevention of spontaneous deviations, and audit and restrict unintended communication paths that can turn local failures into collective loss of control.

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-09-17 in cs.CR).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2609.18460 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T13:56:52.905Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-09-17?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).