Metraipolis › archive › dc_ddc7616aee02

VagueBench: Benchmarking the Robustness of Embodied Agents on Ambient Context Illusions — arXiv:2611.03765

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.03765
License (as archived)
CC0-1.0
Archived
2026-11-04T12:27:28.800Z
SHA-256 of the archived text
8c9f94c630b6db3f9ea24b668ea260985d2ed536274086d0fa810a05052515d1
Stable URL of this page
https://metraipolis.com/documento/dc_ddc7616aee02

How to cite

VagueBench: Benchmarking the Robustness of Embodied Agents on Ambient Context Illusions — arXiv:2611.03765 — arXiv (en), https://arxiv.org/abs/2611.03765 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ddc7616aee02 (Metraipolis archive dc_ddc7616aee02, sha256 8c9f94c630b6db3f9ea24b668ea260985d2ed536274086d0fa810a05052515d1)
BibTeX
@misc{dcddc7616aee02,
  title = {VagueBench: Benchmarking the Robustness of Embodied Agents on Ambient Context Illusions — arXiv:2611.03765},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.03765},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ddc7616aee02; sha256 8c9f94c630b6db3f9ea24b668ea260985d2ed536274086d0fa810a05052515d1},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.03765. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Embodied agents are increasingly deployed in open-world environments, yet current robustness evaluations remain biased toward hazardous instructions and jailbreak vulnerabilities. A more fundamental issue is largely overlooked: agents are rarely tested under semantically sufficient but contextually minimal instructions (i.e., common in real-world human communication), where auxiliary contextual cues are absent. For the first, we identify a novel phenomenon termed Ambient Context Illusion, where agents fail to execute tasks correctly when non-essential contextual details are omitted, despite unchanged task semantics. To systematically study this, we establish VagueBench, a benchmark built upon the AI2-THOR interactive simulation environment for the evaluation of context dependency in embodied agents. We systematically consider key semantic components of instructions (entities, procedures, constraints, and prior knowledge) and derive a unified taxonomy with four categories and nine subtypes to reduce auxiliary information. VagueBench contains 1,303 instruction pairs across diverse indoor environments, constructed via a human-agent co-generation framework that integrates semantic redundancy estimation, taxonomy-guided transformation, and rigorous semantic-invariance verification. Through extensive evaluation on 6 state-of-the-art embodied agents, we observe significant degradation on VagueBench in both task performance and executability (relatively -19.65% in task success rate and relatively +54.10% in empty plan rate) revealing a strong reliance on auxiliary linguistic signals rather than core task semantics. Furthermore, we demonstrate that this phenomenon persists in real-world robotic systems, where it can lead to incorrect decisions and potential safety risks.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.AI, nuovi lavori del giorno).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2611.03765 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-04T12:26:47.101Z dalla pagina https://arxiv.org/list/cs.AI/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).