Metraipolis › archive › dc_2f07d3070549
When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation — arXiv:2609.36685
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.36685
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T18:38:21.305Z
- SHA-256 of the archived text
d2e627b78966ffde2e5e0b16ccf01a4364b000bd0b669071aa4fe67dd2787439- Stable URL of this page
https://metraipolis.com/documento/dc_2f07d3070549
How to cite
When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation — arXiv:2609.36685 — arXiv (en), https://arxiv.org/abs/2609.36685 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2f07d3070549 (Metraipolis archive dc_2f07d3070549, sha256 d2e627b78966ffde2e5e0b16ccf01a4364b000bd0b669071aa4fe67dd2787439)
BibTeX
@misc{dc2f07d3070549,
title = {When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation — arXiv:2609.36685},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.36685},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2f07d3070549; sha256 d2e627b78966ffde2e5e0b16ccf01a4364b000bd0b669071aa4fe67dd2787439},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.36685. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Co-speech gesture generation aims to synthesize natural gestures that are both temporally synchronized with speech and semantically consistent with the spoken content. Although recent methods can generate rhythmically plausible motions, they often rely heavily on acoustic prosody while underutilizing textual semantics, especially when semantic annotations are incomplete, noisy, or unavailable. Consequently, the generated gestures may follow speech rhythm while failing to express the intended semantics. To address this problem, we propose a reliability-aware semantic-rhythm control framework for co-speech gesture generation. We first learn a discrete motion prior that represents continuous gestures in a compact and structured motion-code space. We then introduce a dual-branch semantic contribution estimation mechanism consisting of a full multimodal branch and an audio-only branch. Their distributional discrepancy is formulated as conditional information gain to quantify how much textual semantics changes the predicted motion. Based on this estimate, a controllable semantic-rhythm objective selectively strengthens semantic guidance in content-relevant segments while limiting unnecessary semantic intervention in rhythm-dominant segments. Furthermore, we treat background noise as an acoustic reliability condition and introduce noise-conditioned feature modulation together with beneficial latent perturbation to improve generation robustness under realistic acoustic environments. Experiments on benchmark datasets demonstrate that the proposed framework achieves a favorable balance among semantic expressiveness, rhythmic synchronization, motion diversity, and robustness, enabling reliable and controllable co-speech gesture generation. Tema degli esploratori: statistica e scienza dei dati (retro-catalogo arXiv: annunci del 2026-09-30 in cs.CV). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2609.36685 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T18:04:24.733Z dalla pagina https://arxiv.org/catchup/cs.CV/2026-09-30?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).