Metraipolis › archive › dc_59ee8707ab84

EMPIRE: Explicit Manipulation Planning as a Learnable Intermediate Representation for Egocentric Hand-Motion Forecasting — arXiv:2608.22449

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.22449
License (as archived)
CC0-1.0
Archived
2026-10-18T07:44:51.504Z
SHA-256 of the archived text
e0f4eb7949c47f67852a6825a5d5da84d9be6a0d03a845b12fbb6e8523a7e549
Stable URL of this page
https://metraipolis.com/documento/dc_59ee8707ab84

How to cite

EMPIRE: Explicit Manipulation Planning as a Learnable Intermediate Representation for Egocentric Hand-Motion Forecasting — arXiv:2608.22449 — arXiv (en), https://arxiv.org/abs/2608.22449 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_59ee8707ab84 (Metraipolis archive dc_59ee8707ab84, sha256 e0f4eb7949c47f67852a6825a5d5da84d9be6a0d03a845b12fbb6e8523a7e549)
BibTeX
@misc{dc59ee8707ab84,
  title = {EMPIRE: Explicit Manipulation Planning as a Learnable Intermediate Representation for Egocentric Hand-Motion Forecasting — arXiv:2608.22449},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.22449},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_59ee8707ab84; sha256 e0f4eb7949c47f67852a6825a5d5da84d9be6a0d03a845b12fbb6e8523a7e549},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.22449. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Forecasting dexterous hand motions from egocentric observations is fundamental to intelligent interactive systems. Existing VLM-based methods typically map observations directly to future motions, overlooking the underlying manipulation process that governs hand-object interactions. Moreover, end-to-end optimization couples manipulation learning with motion synthesis, causing motion-generation gradients to interfere with the pre-learned manipulation-aware representations. To overcome these limitations, we propose EMPIRE, a two-stage framework that introduces Explicit Manipulation Planning as an Intermediate Representation for Egocentric hand-motion forecasting. Stage I: Learn to Plan. EMPIRE first learns explicit manipulation plans from multimodal context to capture the progression of hand-object interactions. Stage II: Learn to Act. A motion generator synthesizes future bimanual hand motions conditioned on frozen planner representations, preventing motion-generation gradients from affecting manipulation planning. To support our method, we further construct EMPIRE-651K, a bimanual hand-motion forecasting dataset comprising 650,910 training windows across 111 tasks, each paired with an explicit per-hand manipulation plan. Under identical training and evaluation protocols, EMPIRE achieves state-of-the-art forecasting accuracy, with an MPJPE of 84.53 mm and a finger-relative error of 38.97mm. We release the code and dataset at this https URL .

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-25 in cs.AI).
Categorie arXiv: Robotics (cs.RO) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2608.22449 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T07:37:52.880Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-08-25?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).