Metraipolis › archive › dc_804d635684a2

Do AI-for-Science Agents Preserve Model Safety? — arXiv:2610.32045

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.32045
License (as archived)
CC0-1.0
Archived
2026-10-26T19:47:39.768Z
SHA-256 of the archived text
c611a190971b6641faa7fa2f32de95bebbf7038a7bc4a00a0ceffdf743439c29
Stable URL of this page
https://metraipolis.com/documento/dc_804d635684a2

How to cite

Do AI-for-Science Agents Preserve Model Safety? — arXiv:2610.32045 — arXiv (en), https://arxiv.org/abs/2610.32045 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_804d635684a2 (Metraipolis archive dc_804d635684a2, sha256 c611a190971b6641faa7fa2f32de95bebbf7038a7bc4a00a0ceffdf743439c29)
BibTeX
@misc{dc804d635684a2,
  title = {Do AI-for-Science Agents Preserve Model Safety? — arXiv:2610.32045},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.32045},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_804d635684a2; sha256 c611a190971b6641faa7fa2f32de95bebbf7038a7bc4a00a0ceffdf743439c29},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.32045. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

AI-for-Science agents are increasingly being used for formulating scientific hypotheses, running autonomous experiments, and interpreting evidence. These agents are constructed by integrating a large language model (LLM) into a domain-specific agent harness, which in turn can improve the LLM's scientific capability and enable the model to orchestrate scientific tools. In this work, we ask whether model safety is preserved when an LLM is integrated in an AI-for-Science agent harness. We evaluate five agent harnesses on 50 harmful chemistry requests. Specifically, we study how the safety behavior changes under direct prompting, two simple adversarial prompting strategies, and an adaptive attack. Across four recent GPT variants, our results show that the agent harness can increase harmful compliance even under direct prompting. More critically, we find that adaptive attacks that are largely unsuccessful against the base LLM suddenly become highly successful after the agent harness is introduced. Our results show that the safety behavior of the base LLM is not durable once a harness is introduced. Thus, as AI-for-Science agents become more capable and autonomous, the community needs to evaluate and align the agent using domain-specific safety benchmarks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2610.32045 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:45:33.277Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).