Metraipolis › archive › dc_ebc0ab0281ee

MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents — arXiv:2608.07533

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.07533
License (as archived)
CC0-1.0
Archived
2026-10-19T09:01:04.388Z
SHA-256 of the archived text
5672c82a3d5696562176e088dfb5f627754d2fc310c85c90a4465971175c4bdc
Stable URL of this page
https://metraipolis.com/documento/dc_ebc0ab0281ee

How to cite

MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents — arXiv:2608.07533 — arXiv (en), https://arxiv.org/abs/2608.07533 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ebc0ab0281ee (Metraipolis archive dc_ebc0ab0281ee, sha256 5672c82a3d5696562176e088dfb5f627754d2fc310c85c90a4465971175c4bdc)
BibTeX
@misc{dcebc0ab0281ee,
  title = {MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents — arXiv:2608.07533},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.07533},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ebc0ab0281ee; sha256 5672c82a3d5696562176e088dfb5f627754d2fc310c85c90a4465971175c4bdc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.07533. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

An embodied agent is an intelligent entity that interacts with its environment through a physical body. Currently, the evaluation of embodied agents primarily relies on two paradigms: (1) manually annotated Visual Question Answering (VQA) pairs and (2) high-level task completion metrics, such as success in navigation or manipulation. The former is labor-intensive and subject to variability in annotation quality. The latter may obscure critical vulnerabilities, allowing agents to complete tasks through suboptimal means or safety violations, thereby concealing safety risks and inefficiencies. Given that spatial cognition is the cornerstone for executing embodied tasks, there is a pressing need to assess whether embodied agents possess robust spatial cognition during task execution. Inspired by metamorphic testing principles in software engineering, we propose MetaSpace, a novel framework designed to evaluate the spatial cognition of agents. By leveraging spatiotemporal multimodal states derived from real execution trajectories, MetaSpace automatically generates test cases based on predefined metamorphic relations (MRs) grounded in logical rules and physical laws. Crucially, we encode these MRs as executable rules in a logic programming language (Prolog). Violations of these relations indicate failures in spatial cognition. Our empirical evaluation across three embodied scenarios demonstrates that MetaSpace successfully detects 90,422 spatial cognition errors in state-of-the-art (SOTA) MLLM-driven agents. We introduce the Spatial Cognition (SC) score to quantify performance. Results indicate that all SOTA agents achieve average scores between 0.44 and 0.52, significantly lower than the human benchmark of 0.96.

Tema degli esploratori: calcolo e infrastrutture (retro-catalogo arXiv: annunci del 2026-08-11 in cs.SE).
Categorie arXiv: Artificial Intelligence (cs.AI) ; Software Engineering (cs.SE).
Pagina del lavoro: https://arxiv.org/abs/2608.07533 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-19T09:00:31.504Z dalla pagina https://arxiv.org/catchup/cs.SE/2026-08-11?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).