Metraipolis › archive › dc_bb62e8f952a2

MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant — arXiv:2609.13076

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.13076
License (as archived)
CC0-1.0
Archived
2026-10-16T03:03:48.548Z
SHA-256 of the archived text
d37d96296cbac83d80826bd0dbab82c1008c976be711233d4f17e69fc6f8330e
Stable URL of this page
https://metraipolis.com/documento/dc_bb62e8f952a2

How to cite

MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant — arXiv:2609.13076 — arXiv (en), https://arxiv.org/abs/2609.13076 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bb62e8f952a2 (Metraipolis archive dc_bb62e8f952a2, sha256 d37d96296cbac83d80826bd0dbab82c1008c976be711233d4f17e69fc6f8330e)
BibTeX
@misc{dcbb62e8f952a2,
  title = {MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant — arXiv:2609.13076},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.13076},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bb62e8f952a2; sha256 d37d96296cbac83d80826bd0dbab82c1008c976be711233d4f17e69fc6f8330e},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.13076. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Conversational voice agents have advanced significantly, offering increasingly natural human-machine interactions through both cascaded and end-to-end architectures. However, while recent benchmarks extensively evaluate dyadic interactions and passive audio comprehension, they largely overlook a prevalent real-world scenario: multi-party conversations. Evaluating agents in these settings is fundamentally more challenging than in dyadic interactions due to the exponentially greater conversational complexity. For voice agents to integrate seamlessly into human group dynamics, they must not only generate contextually appropriate responses but also demonstrate a nuanced understanding of open turn-taking. To address this gap, we introduce Multiparty Bench (MP-Bench), the first benchmark specifically designed to objectively evaluate conversational speech systems as active participants within multi-party contexts. MP-Bench assesses agent behavior along two primary dimensions: turn-taking awareness and response appropriateness. Additionally, we incorporate comprehension-based question-answering tasks as a complementary evaluation. By benchmarking 12 voice agents, we find that real-time voice agents stay at or below 22% on multiparty comprehension and remain near chance on multiparty turn-taking, exposing an open challenge for real-time voice agents under multiparty scenario.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-14 in cs.AI).
Categorie arXiv: Audio and Speech Processing (eess.AS) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.13076 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T02:51:27.980Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-14?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).