Metraipolis › archive › dc_2d3f9dfb38ef

DiaHeads: Jointly Generating Coordinated 3D Dialogue Heads with Non-verbal Cues from Dual Audio — arXiv:2611.06970

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.06970
License (as archived)
CC0-1.0
Archived
2026-11-06T14:54:17.704Z
SHA-256 of the archived text
d9983801efcf51f14ffeaefebe9ab28bf2f99fcc36b0e03ca6b8197372924089
Stable URL of this page
https://metraipolis.com/documento/dc_2d3f9dfb38ef

How to cite

DiaHeads: Jointly Generating Coordinated 3D Dialogue Heads with Non-verbal Cues from Dual Audio — arXiv:2611.06970 — arXiv (en), https://arxiv.org/abs/2611.06970 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2d3f9dfb38ef (Metraipolis archive dc_2d3f9dfb38ef, sha256 d9983801efcf51f14ffeaefebe9ab28bf2f99fcc36b0e03ca6b8197372924089)
BibTeX
@misc{dc2d3f9dfb38ef,
  title = {DiaHeads: Jointly Generating Coordinated 3D Dialogue Heads with Non-verbal Cues from Dual Audio — arXiv:2611.06970},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.06970},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2d3f9dfb38ef; sha256 d9983801efcf51f14ffeaefebe9ab28bf2f99fcc36b0e03ca6b8197372924089},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.06970. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Audio-driven 3D head animation has made considerable progress in generation realism, yet generating coordinated behavior for two-person dialogue remains difficult, because current dialogue head generation methods overlook the modeling of non-verbal behavior semantics between the two participants. To address this limitation, we present DiaHeads, a framework that generates the behavior of both participants jointly. The two audio streams are concatenated and decoded together, so each participant's motion is conditioned on the other, which makes the interaction part of the generation itself. We further introduce four discrete non-verbal cues, gaze-on, head shaking, nodding, and smiling, and fuse them with the continuous parameters in a shared latent space through a product-of-experts posterior. These cues carry the interaction semantics and constrain how the continuous motion is generated. A flow-based prior over the shared latent variable replaces the standard Gaussian prior and makes the latent motion temporally dependent. These designs therefore yield coordinated 3D dialogue heads. To support this task we build CANDOR-FG, a dialogue corpus derived from CANDOR and augmented with synchronized FLAME parameters and 3D gaze for both interlocutors. Experiments on CANDOR-FG show that DiaHeads achieves competitive per-participant fidelity and substantial gains in interpersonal coordination over single-speaker and dyadic baselines. In addition, ablations confirm that joint generation, the flow-based prior, and multi-cue supervision are the sources of this coordination. Finally, a user study shows that the advantage of DiaHeads comes from perceived interactional coordination rather than from raw motion fidelity alone.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.MM, nuovi lavori del giorno).
Categorie arXiv: Multimedia (cs.MM).
Pagina del lavoro: https://arxiv.org/abs/2611.06970 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-06T14:26:42.599Z dalla pagina https://arxiv.org/list/cs.MM/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).