Metraipolis › archive › dc_5b81da79bb64
CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation — arXiv:2609.31418
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.31418
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T03:15:04.064Z
- SHA-256 of the archived text
47dbb20a90c7d5d3a75c94d2fb056cdd29b3ccebbda875d357d1f3ebc61e207f- Stable URL of this page
https://metraipolis.com/documento/dc_5b81da79bb64
How to cite
CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation — arXiv:2609.31418 — arXiv (en), https://arxiv.org/abs/2609.31418 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5b81da79bb64 (Metraipolis archive dc_5b81da79bb64, sha256 47dbb20a90c7d5d3a75c94d2fb056cdd29b3ccebbda875d357d1f3ebc61e207f)
BibTeX
@misc{dc5b81da79bb64,
title = {CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation — arXiv:2609.31418},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.31418},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5b81da79bb64; sha256 47dbb20a90c7d5d3a75c94d2fb056cdd29b3ccebbda875d357d1f3ebc61e207f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31418. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
A photorealistic 3D view tells a teleoperator where a robot is, but not what the scene contains, how well each object has been observed, or how to turn pointing and speech into robot action. CognitiveReality turns a robot's RGB-D stream into a live, semantically indexed Gaussian-TSDF map shared by an operator in virtual reality and a tool-using language agent. One mapper binary serves any platform through configuration alone: it ingests poses from robot SLAM, joint kinematics, motion capture or an inline visual tracker, bridges localization outages with a shadow tracker and keyframe-anchored PnP, and maintains open-vocabulary instance identities with per-object quality at 2 Hz. Speech and controller rays are grounded against persistent scene objects through validated typed tools and operator-confirmed robot actions. In the controlled agent evaluation, the deployed local Qwen3-VL-8B router reaches 81.24\% tool exact match, while merge-aware replay correctly redirects 101 absorbed object identifiers. On robot data CognitiveReality exceeds a Gaussian-plus-SDF baseline by 2-8 dB; pose error through 5-40 s SLAM outages stays within 1-8 cm. Deployed live on two quadrupeds, the agent executed 26 of 30 navigation requests and 20 of 20 re-observation requests, raising object quality by 2-5 dB. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-28 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.31418 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T02:58:08.999Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).