Metraipolis › archive › dc_e8cbaf2bc984
RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents — arXiv:2609.25636
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.25636
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T16:43:28.923Z
- SHA-256 of the archived text
e006490e048bb4cbe552d0c7f4584dfc727e2bd24b50a66ac9d50a45168dd06d- Stable URL of this page
https://metraipolis.com/documento/dc_e8cbaf2bc984
How to cite
RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents — arXiv:2609.25636 — arXiv (en), https://arxiv.org/abs/2609.25636 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_e8cbaf2bc984 (Metraipolis archive dc_e8cbaf2bc984, sha256 e006490e048bb4cbe552d0c7f4584dfc727e2bd24b50a66ac9d50a45168dd06d)
BibTeX
@misc{dce8cbaf2bc984,
title = {RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents — arXiv:2609.25636},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.25636},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_e8cbaf2bc984; sha256 e006490e048bb4cbe552d0c7f4584dfc727e2bd24b50a66ac9d50a45168dd06d},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25636. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Modern embodied agents achieve impressive success rates, yet their actual instruction-following ability is far weaker than these numbers suggest. We trace this illusion to a structural property we term low scene entropy: when a visual scene admits only one valid task, language becomes redundant and a policy can score highly while barely using it. We introduce RoboFollow, a diagnostic benchmark with three principles: (1) High Scene Entropy: each training scene supports multiple kinematically distinct task branches, making vision alone insufficient and forcing reliance on language. (2) Hierarchical Diagnostic Protocol: a four-level protocol (L0--L3) progressively perturbs visual layout and semantics, probing whether equivalent instructions yield consistent behavior and distinct ones yield discriminable behavior across spatial relations, attributes, trajectory constraints, and logic. (3) Confound-Controlled Diagnosis: we simplify interaction objects, restrict actions to the trained repertoire and report stage-wise Intent and Execution scores, isolating comprehension from motor execution. Evaluation of nine VLA and WAM policies shows that strong L0 performance, where attained, does not reliably transfer to L1--L3 under our fine-tuning setup. Representative mitigations, including stronger VLM backbones, QA co-training, LangForce, and Classifier-Free Guidance, all fail to close this gap. RoboFollow exposes genuine instruction following as a critical, overlooked bottleneck. Code and dataset are available at this https URL and this https URL . Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-23 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.25636 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T15:56:53.444Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).