Metraipolis › archive › dc_09f2befc796f
PACT: End-to-End Learning of Human Pose, Contacts, and Forces from Video — arXiv:2610.00451
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.00451
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-06T12:09:44.254Z
- SHA-256 of the archived text
28bf3fa97521385531bd15362c05ded212e3c4331526cbd78d03faf838d3c255- Stable URL of this page
https://metraipolis.com/documento/dc_09f2befc796f
How to cite
PACT: End-to-End Learning of Human Pose, Contacts, and Forces from Video — arXiv:2610.00451 — arXiv (en), https://arxiv.org/abs/2610.00451 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_09f2befc796f (Metraipolis archive dc_09f2befc796f, sha256 28bf3fa97521385531bd15362c05ded212e3c4331526cbd78d03faf838d3c255)
BibTeX
@misc{dc09f2befc796f,
title = {PACT: End-to-End Learning of Human Pose, Contacts, and Forces from Video — arXiv:2610.00451},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.00451},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_09f2befc796f; sha256 28bf3fa97521385531bd15362c05ded212e3c4331526cbd78d03faf838d3c255},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.00451. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Human motion, environmental contacts, and interaction forces are governed by common physical laws, yet existing approaches typically separate visual pose reconstruction from contact and force estimation. This separation limits joint reasoning and can propagate errors between stages. We introduce PACT, an end-to-end model that jointly learns to estimate human pose, contacts and contact forces from monocular video. Our approach augments a human reconstruction foundation model with learnable contact-force tokens and a temporal transformer that integrates visual features with world-space motion. Joint prediction heads refine human poses and estimate contacts and forces, while physics-based supervision encourages consistency between the reconstructed motion and interaction forces. To address the scarcity of force annotations, we develop a data annotation pipeline that combines contact labeling with physics-based motion and force optimization, producing training supervision from synthetic and real-world videos. We also introduce a real-world climbing benchmark ForceWall with climbing videos and corresponding ground-truth contact forces obtained from the force sensors. Experiments demonstrate state-of-the-art contact and force estimation, outperforming staged reconstruction approaches and generalizing to interactions beyond the training distribution. These results support end-to-end joint learning as an effective approach to recovering human motion and physical interactions from video. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-02 in cs.AI). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG). Pagina del lavoro: https://arxiv.org/abs/2610.00451 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-06T10:21:28.074Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).