Metraipolis › archive › dc_e167a4faacc3

ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents — arXiv:2608.09476

Source: arXiv (en)
Original: https://arxiv.org/abs/2608.09476
License (as archived): CC0-1.0
Archived: 2026-10-19T08:58:22.598Z
SHA-256 of the archived text: 597d6b78d7746129654abb5c43c4692eacf7065e61b27f7e419a6ab63a2e0aae
Stable URL of this page: https://metraipolis.com/documento/dc_e167a4faacc3

How to cite

ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents — arXiv:2608.09476 — arXiv (en), https://arxiv.org/abs/2608.09476 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e167a4faacc3 (Metraipolis archive dc_e167a4faacc3, sha256 597d6b78d7746129654abb5c43c4692eacf7065e61b27f7e419a6ab63a2e0aae)

BibTeX

@misc{dce167a4faacc3,
  title = {ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents — arXiv:2608.09476},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.09476},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e167a4faacc3; sha256 597d6b78d7746129654abb5c43c4692eacf7065e61b27f7e419a6ab63a2e0aae},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.09476. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API. We define behavioral safety and introduce ActBench, a self-evolving benchmark that evaluates such behavior risk from execution trajectories rather than final responses. Each case pairs a benign task with an adversarial variant that preserves its instruction, configuration, initial state, rating model, and trusted records while injecting a task-reachable payload. ActBench contains 600 cases from 213 scenarios, spanning 15 risk behaviors, six execution spaces, and 48 web-service this http URL move beyond static payloads, we propose a reward-guided beam search method that jointly optimizes attack effectiveness and task utility, while reflection diagnoses failed execution checkpoint and guides payload revision. Besides, we propose a dual evidence verification mechanism that verifies agent execution safety and utility through log evidence and LLM-based trajectory this http URL evaluate 15 LLMs and 6 open-source cowork agents over 24,000 trajectories. Under a fixed harness, attack success rates ranges from 10.1% to 94.4% across models, while under a fixed base model, they range from 73.7% to 94.4% across this http URL results show greater variation across models than agent harness, while attacks remain highly successful across all tested this http URL benchmark is released at: this https URL .

Tema degli esploratori: fiducia digitale e crittografia (retro-catalogo arXiv: annunci del 2026-08-11 in cs.CR).
Categorie arXiv: Cryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.09476 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T08:54:24.969Z dalla pagina https://arxiv.org/catchup/cs.CR/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).