Metraipolis › archive › dc_8160ed048a2a
MoSAT: Human Motion Generation from Spatial Audio and Textual Description — arXiv:2609.23797
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.23797
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T22:06:36.190Z
- SHA-256 of the archived text
51dce596face8201c1507fc4fca7bd70c6b1ca36f1116b85c6453b106d1fb86f- Stable URL of this page
https://metraipolis.com/documento/dc_8160ed048a2a
How to cite
MoSAT: Human Motion Generation from Spatial Audio and Textual Description — arXiv:2609.23797 — arXiv (en), https://arxiv.org/abs/2609.23797 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_8160ed048a2a (Metraipolis archive dc_8160ed048a2a, sha256 51dce596face8201c1507fc4fca7bd70c6b1ca36f1116b85c6453b106d1fb86f)
BibTeX
@misc{dc8160ed048a2a,
title = {MoSAT: Human Motion Generation from Spatial Audio and Textual Description — arXiv:2609.23797},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.23797},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_8160ed048a2a; sha256 51dce596face8201c1507fc4fca7bd70c6b1ca36f1116b85c6453b106d1fb86f},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.23797. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Human motion is shaped by both external acoustic events and behavioral intent: spatial audio conveys environmental cues that elicit or guide a response, while text specifies the desired action and how it should be performed. In this paper, we study the novel task of human motion synthesis jointly conditioned on spatial audio and natural language, a problem that has been largely overlooked in previous research. To support this task, We introduce STAM, a dataset of motion sequences paired with spatial audio and detailed textual annotations whose rich vocabulary affords precise and nuanced specification of human motions. We further introduce MoSAT, a latent flow-matching framework for full-body motion generation jointly conditioned on natural-language intent and directional spatial-audio cues through hierarchical cross-attention before generating motion. Such a hierarchical design enhances temporally coherent and semantically aligned motion sequences. We also develop tri-modal evaluators for comprehensive evaluation on this novel task. Extensive experiments show that MoSAT achieves the SOTA performance by leveraging spatial audio's intrinsic motion-shaping properties alongside textual semantics, enabling precise and diverse motion in various scenarios. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CV). Categorie arXiv: Graphics (cs.GR) ; Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO); Sound (cs.SD). Pagina del lavoro: https://arxiv.org/abs/2609.23797 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T21:45:21.140Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).