Metraipolis › archive › dc_dc3a080241d8

Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation — arXiv:2609.31186

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.31186
License (as archived): CC0-1.0
Archived: 2026-10-08T01:38:37.672Z
SHA-256 of the archived text: 6530372cb2bdfb4c02b9dc20d876a9965fb0a3e31cae8d64b87d5367f9960695
Stable URL of this page: https://metraipolis.com/documento/dc_dc3a080241d8

How to cite

Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation — arXiv:2609.31186 — arXiv (en), https://arxiv.org/abs/2609.31186 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc3a080241d8 (Metraipolis archive dc_dc3a080241d8, sha256 6530372cb2bdfb4c02b9dc20d876a9965fb0a3e31cae8d64b87d5367f9960695)

BibTeX

@misc{dcdc3a080241d8,
  title = {Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation — arXiv:2609.31186},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31186},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc3a080241d8; sha256 6530372cb2bdfb4c02b9dc20d876a9965fb0a3e31cae8d64b87d5367f9960695},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31186. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Artificial intelligence is advancing rapidly, with increasingly capable systems taking larger roles in reasoning, decision-making, scientific discovery, and autonomous development. As AI begins to participate in its own improvement, from model training and experience accumulation to agent evolution and automated AI development, the prospect of recursive self-improvement (RSI) is becoming increasingly relevant. This transition raises a fundamental safety question: how can safety be maintained when the system, its accumulated experience, and even the process producing its successors continue to change? We introduce Evolutionary Safety as a perspective for studying safety under persistent and recursive self-improvement. It concerns not only whether an AI system is safe at a particular moment, but how safety properties change, persist, accumulate, and propagate throughout evolution. We characterize recurring manifestations, including intent drift, error accumulation, experience contamination, safety-property erosion, evaluator drift, and risk propagation. We then develop a taxonomy spanning persistent agent state, model state, evaluation and environmental feedback, computational substrate, and meta-level update mechanisms. Building on this taxonomy, we examine how evolutionary risks can be discovered and evaluated across states, updates, trajectories, and lineages, and derive governance principles for modification, selection, authorization, provenance, and recovery. Finally, we outline open problems toward maintaining safety guarantees as AI systems become increasingly persistent, adaptive, and recursively self-improving. Project resources and proposed evaluation systems are available at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.31186 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T01:12:14.357Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).