Metraipolis › archive › dc_dc0fee971388
Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos — arXiv:2609.34630
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.34630
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T23:52:19.184Z
- SHA-256 of the archived text
cdf568aa54e2215af673c0ce1ed691afa53e386b12781b9e5f0f3001aa15032f- Stable URL of this page
https://metraipolis.com/documento/dc_dc0fee971388
How to cite
Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos — arXiv:2609.34630 — arXiv (en), https://arxiv.org/abs/2609.34630 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dc0fee971388 (Metraipolis archive dc_dc0fee971388, sha256 cdf568aa54e2215af673c0ce1ed691afa53e386b12781b9e5f0f3001aa15032f)
BibTeX
@misc{dcdc0fee971388,
title = {Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos — arXiv:2609.34630},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.34630},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dc0fee971388; sha256 cdf568aa54e2215af673c0ce1ed691afa53e386b12781b9e5f0f3001aa15032f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.34630. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Real-world AI systems must reason about objects that are no longer visible: an AR assistant guiding a user back to an object used earlier, a household robot retrieving an item someone put away. This requires not just recalling where an object was last seen, but updating its state when it is moved and retaining that update once it leaves view. We refer to this as out-of-sight spatiotemporal reasoning. We introduce Beyond3D, the first VQA benchmark to isolate this ability in dynamic egocentric video: every query targets an object that has been relocated and has since left the field of view. We create our questions from HD-EPIC annotations, building a visibility track for each dynamic object from its 3D position, the camera pose, and the scene geometry to understand at each moment whether it is visible, occluded, or out of view. Beyond3D comprises 9,000 questions in eight types over 135 videos from nine participants, organized as one reasoning chain: visual grounding (is the target observable now), temporal grounding (when it was last visible and last placed), scene localization (which fixture anchors that location), and 3D spatial perception (where it lies relative to the current viewpoint or another object in the scene). We benchmark nine general-purpose and spatially specialized VLMs. The best model reaches 42.2% against 29.7% chance and text-only baselines reaching 31.9%, with the largest failures in recovering when an object was last visible, showing that tracking object movement out of sight remains far from solved for current VLMs. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-29 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.34630 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T23:29:27.416Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).