Metraipolis › archive › dc_0fe863f87ff3

VT-MUSE: Multimodal Unified Sequential Visuotactile Representation Learning for Manipulation — arXiv:2608.21290

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.21290
License (as archived)
CC0-1.0
Archived
2026-10-18T10:56:33.383Z
SHA-256 of the archived text
fd02cad139a612f7459330bfdccbee503746b8420a826b4ef048e058190068f0
Stable URL of this page
https://metraipolis.com/documento/dc_0fe863f87ff3

How to cite

VT-MUSE: Multimodal Unified Sequential Visuotactile Representation Learning for Manipulation — arXiv:2608.21290 — arXiv (en), https://arxiv.org/abs/2608.21290 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_0fe863f87ff3 (Metraipolis archive dc_0fe863f87ff3, sha256 fd02cad139a612f7459330bfdccbee503746b8420a826b4ef048e058190068f0)
BibTeX
@misc{dc0fe863f87ff3,
  title = {VT-MUSE: Multimodal Unified Sequential Visuotactile Representation Learning for Manipulation — arXiv:2608.21290},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.21290},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_0fe863f87ff3; sha256 fd02cad139a612f7459330bfdccbee503746b8420a826b4ef048e058190068f0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.21290. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

We propose VT-MUSE, a Multimodal Unified SEquential representation learning framework for visuotactilemanipulation. Existing approaches often encode visual and tactile observations independently before fusion, limiting their ability to capture fine-grained cross-modal dependencies. Moreover, most methods focus on observations at the current time step and overlook the temporal evolution of contact. VT-MUSE addresses both limitations through a two-stage representation learning framework. In Stage I, modality specific encoders are jointly adapted via cross-modal temporal alignment and masked-view consistency. In Stage II, a conditional variational latent model processes masked visual sequences together with full tactile histories. Auxiliary decoders reconstruct the masked recent visual observations and predict tactile depth changes, encouraging the latent representation to retain both global visual context and local contact dynamics. The learned representation is subsequently integrated into a lightweight Transformer policy through gated cross-attention. On the simulation benchmark, VT-MUSE outperforms the strongest baseline evaluated on all tasks by 11 percentage points and also achieves substantial improvements in real-world experiments.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-08-24 in cs.CV).
Categorie arXiv: Robotics (cs.RO) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2608.21290 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T10:48:13.627Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-08-24?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).