Metraipolis › archive › dc_66824ec2bb4a

Closing the Affective Loop: Multimodal Speaker-Listener Emotion-Dynamics-Aware Empathetic Social Robots — arXiv:2608.16686

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.16686
License (as archived)
CC0-1.0
Archived
2026-10-18T19:01:32.509Z
SHA-256 of the archived text
5e7a9c0d342202255124a224b7cd538dfc8b29e9c311537a5c60c715d7997ad0
Stable URL of this page
https://metraipolis.com/documento/dc_66824ec2bb4a

How to cite

Closing the Affective Loop: Multimodal Speaker-Listener Emotion-Dynamics-Aware Empathetic Social Robots — arXiv:2608.16686 — arXiv (en), https://arxiv.org/abs/2608.16686 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_66824ec2bb4a (Metraipolis archive dc_66824ec2bb4a, sha256 5e7a9c0d342202255124a224b7cd538dfc8b29e9c311537a5c60c715d7997ad0)
BibTeX
@misc{dc66824ec2bb4a,
  title = {Closing the Affective Loop: Multimodal Speaker-Listener Emotion-Dynamics-Aware Empathetic Social Robots — arXiv:2608.16686},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.16686},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_66824ec2bb4a; sha256 5e7a9c0d342202255124a224b7cd538dfc8b29e9c311537a5c60c715d7997ad0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.16686. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Empathetic social robots should respond not only to what users say, but also to how their emotions dynamically evolve during interaction. However, existing empathetic dialogue systems are often text-centered and primarily model empathy as a one-way mapping from the user's emotion to the system response, limiting their ability to capture embodied speaker--listener affective exchange. We present AffectLoop, a multimodal speaker-listener emotion-dynamics-aware spoken dialogue system implemented on the Misty II robot. The system tracks the speaker's verbal and facial affective dynamics, estimates the robot listener's own verbal and behavioral affective state, and conditions LLM-based response generation on both affective streams. The robot then generates a short spoken empathetic response together with emotionally congruent embodied behavior, forming a closed speaker--listener affective loop. We evaluate the system in a pilot within-subject study with five participants, comparing it with an otherwise identical utterance-conditioned baseline that omits the speaker- and listener-affective-state inputs. The proposed system received higher overall impression ratings, especially for empathetic response and user satisfaction. Post-hoc log analysis further showed higher speaker-listener affective alignment and stronger valence-based distress recovery. These preliminary results suggest that explicitly modeling both speaker emotional dynamics and listener affective state can improve embodied empathetic interaction.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-08-18 in cs.CL).
Categorie arXiv: Human-Computer Interaction (cs.HC) ; Computation and Language (cs.CL); Robotics (cs.RO).
Pagina del lavoro: https://arxiv.org/abs/2608.16686 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-18T18:06:57.936Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-08-18?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).