Metraipolis › archive › dc_da0dc27261e2
Do Multi-Turn Conversational Capabilities Carry Over from Text to Speech? A Parallel Multilingual Benchmark — arXiv:2611.04390
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.04390
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-04T12:32:54.131Z
- SHA-256 of the archived text
7d1a435db63c8ea8a12a90c2c89abbc235c36b8565351df9c59f8fd6061bf8c6- Stable URL of this page
https://metraipolis.com/documento/dc_da0dc27261e2
How to cite
Do Multi-Turn Conversational Capabilities Carry Over from Text to Speech? A Parallel Multilingual Benchmark — arXiv:2611.04390 — arXiv (en), https://arxiv.org/abs/2611.04390 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_da0dc27261e2 (Metraipolis archive dc_da0dc27261e2, sha256 7d1a435db63c8ea8a12a90c2c89abbc235c36b8565351df9c59f8fd6061bf8c6)
BibTeX
@misc{dcda0dc27261e2,
title = {Do Multi-Turn Conversational Capabilities Carry Over from Text to Speech? A Parallel Multilingual Benchmark — arXiv:2611.04390},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.04390},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_da0dc27261e2; sha256 7d1a435db63c8ea8a12a90c2c89abbc235c36b8565351df9c59f8fd6061bf8c6},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.04390. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Successful multi-turn voice interaction requires models to retain information from earlier turns, follow persistent instructions, and track revisions to conversational state. These abilities are usually evaluated from transcripts, leaving open whether successes demonstrated from text carry over when the same turns are spoken, and whether this transfer is consistent across languages. We introduce Omnilingua-AudioMC, a strictly parallel multilingual extension of Audio MultiChallenge, to study this question. The benchmark contains 354 conversations and 1,404 rubric items across ten languages, with content, turn structure, cross-turn dependencies, and evaluation criteria aligned across transcript and speech conditions. Across six speech-language models, the shift from reference transcripts to direct speech input is strongly model- and language-dependent: models with similar transcript performance can diverge substantially under speech, and their relative strengths across languages can change. Paired rubric-level analysis shows that aggregate transcript-speech gaps can obscure which conversational requirements are preserved, because losses of transcript-demonstrated successes can be offset by gains on other items. Replacing preceding user turns, the current turn, or both with model-generated transcripts localizes recoverable degradation: for the two models with the largest gaps, history substitution yields a larger macro-level improvement than current-turn substitution in all ten languages, while the strongest direct-speech model gains little from either. These results show that transcript performance is an unreliable proxy for multilingual spoken conversational ability and identify the representation of preceding spoken context as an important source of recoverable degradation. Our code is publicly available at this https URL . Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL). Pagina del lavoro: https://arxiv.org/abs/2611.04390 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-04T12:27:02.038Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).