Metraipolis › archive › dc_4eda47a693ef

MotorSense: A Video-EMG Dataset of Motoric Representations for Action Understanding — arXiv:2610.41127

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.41127
License (as archived)
CC0-1.0
Archived
2026-11-03T08:20:26.882Z
SHA-256 of the archived text
ea4e3a2f62e12609cccbbde4dce20153145b163d3bed4c3d63d6801b001a3691
Stable URL of this page
https://metraipolis.com/documento/dc_4eda47a693ef

How to cite

MotorSense: A Video-EMG Dataset of Motoric Representations for Action Understanding — arXiv:2610.41127 — arXiv (en), https://arxiv.org/abs/2610.41127 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4eda47a693ef (Metraipolis archive dc_4eda47a693ef, sha256 ea4e3a2f62e12609cccbbde4dce20153145b163d3bed4c3d63d6801b001a3691)
BibTeX
@misc{dc4eda47a693ef,
  title = {MotorSense: A Video-EMG Dataset of Motoric Representations for Action Understanding — arXiv:2610.41127},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.41127},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4eda47a693ef; sha256 ea4e3a2f62e12609cccbbde4dce20153145b163d3bed4c3d63d6801b001a3691},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.41127. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Computer vision models learn hand action only from its visual consequences, without access to the underlying embodied motor signals. We address this absence with MotorSense, the first large-scale paired egocentric video and surface electromyography (EMG) dataset. MotorSense pairs bimanual EMG bands with egocentric video and yields synchronized video, bimanual EMG and IMU, and audio across 100 hours of unscripted manipulation. EMG offers a powerful scalable alternative to manual annotation: it captures finger movement, hand configuration, and the forces exerted between hand and environment as a dense signal free of occlusion and labeler bias, whereas manual annotation is costly, subjective, and unreliable under occlusion. We use MotorSense to learn sensorimotor alignment, producing an organization of action that is shared across the visual and motoric spaces, through cross-modal pre-training between video and EMG. The resultant vision models are then transferred to downstream tasks that benefit from sensorimotor alignment: in this case, action recognition, 3D hand reconstruction, and 3D contact reconstruction. We further demonstrate a generative application in which EMG motor signals are used to condition a video diffusion model, yielding videos of fine-grained hand manipulation. Across four action-centric computer vision tasks and eight popular video datasets, we demonstrate that sensorimotor alignment provides a stronger foundation for visual representation learning than conventional self-supervised learning methods over video, outperforming every baseline across all tasks and datasets evaluated.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2610.41127 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-03T08:06:38.820Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).