Metraipolis › archive › dc_ef17aa33e7b6
PolyUMI: Accessible Visual-Tactile-Audio Data Collection for Object Inference and Manipulation — arXiv:2609.29760
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.29760
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-08T07:17:02.620Z
- SHA-256 of the archived text
381a569f6b83905c75c3515c018af4b6235417a47599a4dbbce1883a63143482- Stable URL of this page
https://metraipolis.com/documento/dc_ef17aa33e7b6
How to cite
PolyUMI: Accessible Visual-Tactile-Audio Data Collection for Object Inference and Manipulation — arXiv:2609.29760 — arXiv (en), https://arxiv.org/abs/2609.29760 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_ef17aa33e7b6 (Metraipolis archive dc_ef17aa33e7b6, sha256 381a569f6b83905c75c3515c018af4b6235417a47599a4dbbce1883a63143482)
BibTeX
@misc{dcef17aa33e7b6,
title = {PolyUMI: Accessible Visual-Tactile-Audio Data Collection for Object Inference and Manipulation — arXiv:2609.29760},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.29760},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_ef17aa33e7b6; sha256 381a569f6b83905c75c3515c018af4b6235417a47599a4dbbce1883a63143482},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.29760. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Humans typically rely on vision, touch, hearing, and proprioception to perceive contact and adapt their actions during manipulation. Providing robots with comparable responsiveness therefore requires hardware that can retain and use these complementary sensory signals. Most imitation-learning systems, however, observe demonstrations primarily through vision and proprioception, limiting access to contact information that is difficult to infer visually. We present PolyUMI, an open-source platform for scalable visual--tactile--audio demonstration collection and robot deployment. Its lightweight, wireless handheld gripper records synchronized wrist-camera, optical tactile, contact-audio, and proprioceptive observations without requiring a tethered workstation. The same sensing finger can be transferred to the robot end effector, preserving the sensing geometry between demonstration collection and policy execution. To effectively use these heterogeneous observations, we further introduce VisTA, a token-level multimodal policy that integrates information across sensors and time to predict contact-aware robot actions. Experiments spanning object inference, slip control, and contact-rich manipulation show that touch and audio reveal task-relevant information beyond vision and that VisTA is competitive with or outperforms existing multimodal policies. Together, PolyUMI and VisTA provide an accessible pipeline for collecting multimodal demonstrations and learning policies that perceive physical interaction beyond vision. Project Page: this https URL Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-25 in cs.RO). Categorie arXiv: Robotics (cs.RO). Pagina del lavoro: https://arxiv.org/abs/2609.29760 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-08T06:38:53.823Z dalla pagina https://arxiv.org/catchup/cs.RO/2026-09-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).