Metraipolis › archive › dc_8708e045275e

Omni-UBench: Benchmarking Omni-Modal LLM-based Guardrails Across Safety, Utility, and Reliability — arXiv:2610.31713

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.31713
License (as archived)
CC0-1.0
Archived
2026-10-26T19:53:45.318Z
SHA-256 of the archived text
5f16a0ad987e76bbf5fb0885941bfe421c0799219b58cc1b645182e6d2d8d96d
Stable URL of this page
https://metraipolis.com/documento/dc_8708e045275e

How to cite

Omni-UBench: Benchmarking Omni-Modal LLM-based Guardrails Across Safety, Utility, and Reliability — arXiv:2610.31713 — arXiv (en), https://arxiv.org/abs/2610.31713 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8708e045275e (Metraipolis archive dc_8708e045275e, sha256 5f16a0ad987e76bbf5fb0885941bfe421c0799219b58cc1b645182e6d2d8d96d)
BibTeX
@misc{dc8708e045275e,
  title = {Omni-UBench: Benchmarking Omni-Modal LLM-based Guardrails Across Safety, Utility, and Reliability — arXiv:2610.31713},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.31713},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8708e045275e; sha256 5f16a0ad987e76bbf5fb0885941bfe421c0799219b58cc1b645182e6d2d8d96d},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.31713. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Advances in Omni-modal Large Language Models (OLLMs) have enabled increasingly seamless human-AI interaction, but their unified processing of heterogeneous modalities also expands the attack surface, allowing adversaries to exploit richer input compositions to circumvent safety alignment. To counter such risks, omni-modal guardrails have been proposed to detect and filter unsafe queries across arbitrary modalities. However, their capabilities and limitations remain insufficiently understood due to the absence of a comprehensive evaluation benchmark. In this paper, we present Omni-UBench, the first comprehensive framework for omni-modal safety assessment, evaluating guardrail models along three critical dimensions, i.e., safety, utility, and reliability. Omni-UBench curates over 183k samples drawn from 35+ heterogeneous sources spanning text, image, audio, video, and their cross-modal combinations, annotated with safety labels under a unified risk taxonomy. Benchmarking under Omni-UBench on 15 representative OLLMs and omni-modal guardrails yields several key findings, including: (1) modern OLLMs exhibit strong content moderation potential even without guardrail-specific tuning; (2) existing guardrails still struggle to identify cross-modal queries that conceal harmful intent or involve sophisticated jailbreaks; (3) they further exhibit substantial safety hallucination and over-sensitivity to textual patterns, revealing a deficient understanding of cross-modal safety semantics. Our findings provide valuable insights for developing more trustworthy omni-modal guardrails in future research.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Cryptography and Security (cs.CR) ; Computation and Language (cs.CL); Human-Computer Interaction (cs.HC).
Pagina del lavoro: https://arxiv.org/abs/2610.31713 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-26T19:45:48.379Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).