Metraipolis › archive › dc_2c8ba7193aff

ORBIT: A Framework for Multi-Agent Safety and Security Evaluations — arXiv:2609.33102

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.33102
License (as archived)
CC0-1.0
Archived
2026-10-07T20:53:54.915Z
SHA-256 of the archived text
583e5c84c76608c7de4f9e761ba17b7019e706bc16499e330d431c47470f8e32
Stable URL of this page
https://metraipolis.com/documento/dc_2c8ba7193aff

How to cite

ORBIT: A Framework for Multi-Agent Safety and Security Evaluations — arXiv:2609.33102 — arXiv (en), https://arxiv.org/abs/2609.33102 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2c8ba7193aff (Metraipolis archive dc_2c8ba7193aff, sha256 583e5c84c76608c7de4f9e761ba17b7019e706bc16499e330d431c47470f8e32)
BibTeX
@misc{dc2c8ba7193aff,
  title = {ORBIT: A Framework for Multi-Agent Safety and Security Evaluations — arXiv:2609.33102},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.33102},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2c8ba7193aff; sha256 583e5c84c76608c7de4f9e761ba17b7019e706bc16499e330d431c47470f8e32},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.33102. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multi-agent LLM systems are increasingly deployed for complex, long-horizon tasks or emerge as a natural consequence of agents interacting in the wild. Yet they give rise to significant safety and security risks: the flexible protocols that enable task generalization also expose novel threats, from cascading prompt injection to inter-agent collusion. Progress in defending against these threats has been slowed by a lack of shared empirical infrastructure, which forces bespoke environment development for every new defense and makes standardized comparison impossible. Existing evaluations address isolated threat models or single-agent settings, but none jointly vary attack, defense, and architecture across realistic multi-agent environments. To address this gap, we introduce ORBIT, a configurable evaluation framework for empirical multi-agent safety and security research, built on UK AISI's Inspect. ORBIT lets researchers configure communication topologies, memory, scheduling, and agent roles. It supports four threat types and four defense strategies, as well as non-adversarial failures, with a benchmark suite spanning five scenario families covering browser use, computer use, agentic coding, customer service, and cooperative allocation. Our central finding is a gap in defense transferability across threats: per-action defenses that cut a compromised agent's attack success by 60 points on multi-issue coding give no measurable protection against colluding agents, and none of the defenses we tested generalized over all attacks tested. We further demonstrate security-performance tradeoffs and interactions between architecture and defense effectiveness. We make ORBIT available open-source at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.MA).
Categorie arXiv: Multiagent Systems (cs.MA) ; Cryptography and Security (cs.CR).
Pagina del lavoro: https://arxiv.org/abs/2609.33102 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T20:24:41.618Z dalla pagina https://arxiv.org/catchup/cs.MA/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).