Metraipolis › archive › dc_aa848fa351bc

Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation — arXiv:2609.00369

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.00369
License (as archived): CC0-1.0
Archived: 2026-10-17T07:10:27.906Z
SHA-256 of the archived text: bba8862e53ea5bbce12bf5282b3a24453eba880d446007f983b2af4654fe6f78
Stable URL of this page: https://metraipolis.com/documento/dc_aa848fa351bc

How to cite

Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation — arXiv:2609.00369 — arXiv (en), https://arxiv.org/abs/2609.00369 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_aa848fa351bc (Metraipolis archive dc_aa848fa351bc, sha256 bba8862e53ea5bbce12bf5282b3a24453eba880d446007f983b2af4654fe6f78)

BibTeX

@misc{dcaa848fa351bc,
  title = {Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation — arXiv:2609.00369},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.00369},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_aa848fa351bc; sha256 bba8862e53ea5bbce12bf5282b3a24453eba880d446007f983b2af4654fe6f78},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.00369. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Generating co-speech gestures that are temporally coherent, semantically aligned with speech, and grounded with surrounding objects remains challenging. Prior speech-driven gesture models emphasize audio-gesture alignment but do not explicitly account for posture constraints or surrounding objects, failing to capture the inherent correlation between body gestures and the physical space. We present Puppeteer, a posture-aware, object-grounded co-speech gesture diffusion model operating in a causal latent space. We decompose long gestures into structured primitives and learn a causal variational autoencoder that encodes them into temporally ordered latent tokens, each depending only on the past. We then perform conditional diffusion directly in the causal latent space, conditioning on speech signals, motion history, an initial posture reference, and object geometry to synthesize physically consistent gestures. This temporally ordered latent formulation enables explicit temporal control and supports tasks such as gesture in-betweening and gesture completion. To better assess co-speech gesture synthesis beyond existing measures, we introduce new evaluation metrics tailored to this task. We also created SceneGes, the first curated synthetic 3D dataset of embodied co-speech gestures and corresponding 3D objects, enabling object-grounded gesture generation. Experiments show that Puppeteer generates more diverse and temporally synchronized gestures than prior methods, while enabling object-grounded gesture synthesis.

Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-02 in cs.CV).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.00369 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:00:58.899Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-02?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).