Metraipolis › archive › dc_cbd620461679

RoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue — arXiv:2609.16614

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.16614
License (as archived)
CC0-1.0
Archived
2026-10-15T18:35:06.272Z
SHA-256 of the archived text
b911f0757002c5f7cb827f0b4b2012dc8a17c0d6413c5fe1f9ef9dc8e55b56b3
Stable URL of this page
https://metraipolis.com/documento/dc_cbd620461679

How to cite

RoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue — arXiv:2609.16614 — arXiv (en), https://arxiv.org/abs/2609.16614 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_cbd620461679 (Metraipolis archive dc_cbd620461679, sha256 b911f0757002c5f7cb827f0b4b2012dc8a17c0d6413c5fe1f9ef9dc8e55b56b3)
BibTeX
@misc{dccbd620461679,
  title = {RoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue — arXiv:2609.16614},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.16614},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_cbd620461679; sha256 b911f0757002c5f7cb827f0b4b2012dc8a17c0d6413c5fe1f9ef9dc8e55b56b3},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.16614. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speech-to-speech dialogue models increasingly support persona control, yet existing spoken role-playing benchmarks remain largely character-centric and short-horizon. This leaves open whether spoken dialogue models can sustain diverse roles over extended interactions, especially beyond predefined fictional characters. We introduce RoleBreak, an open benchmark for long-horizon role-playing robustness in spoken dialogue. RoleBreak contains 310 character-based and user-centered roles, 6,688 human-verified dialogue turns, and 11,743 fine-grained evaluation criteria, with 1,856 turns carrying expressive emotion targets for evaluating vocal emotion. Its scenarios are designed to stress role consistency, interaction quality, safety, and affect over extended conversations. We evaluate nine configurations spanning full-duplex, omni-modal, and cascaded ASR--LLM--TTS paradigms. We find four key patterns. First, current systems are substantially stronger at semantic role adherence than at vocal emotion. Second, semantic robustness remains brittle over long interactions: even the strongest evaluated system encounters its first persona and safety failures after only 10.4 and 11.6 turns on average. Third, scaling the LLM substantially improves semantic robustness and delays failure, but yields little improvement in vocal emotion. Finally, user vocal emotion affects role-playing behavior even when linguistic content is fixed. These findings highlight persistent gaps in both long-horizon robustness and vocal expressiveness in spoken role-playing systems.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-16 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.16614 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-15T18:03:09.673Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-16?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).