Metraipolis › archive › dc_bf64af22b497

MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents — arXiv:2609.24812

Source: arXiv (en)
Original: https://arxiv.org/abs/2609.24812
License (as archived): CC0-1.0
Archived: 2026-10-09T06:50:13.802Z
SHA-256 of the archived text: 92bee89032dd78551203ac2ff24c281a0f7b2ebfd0a277742d5bc372b2acaf76
Stable URL of this page: https://metraipolis.com/documento/dc_bf64af22b497

How to cite

MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents — arXiv:2609.24812 — arXiv (en), https://arxiv.org/abs/2609.24812 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_bf64af22b497 (Metraipolis archive dc_bf64af22b497, sha256 92bee89032dd78551203ac2ff24c281a0f7b2ebfd0a277742d5bc372b2acaf76)

BibTeX

@misc{dcbf64af22b497,
  title = {MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents — arXiv:2609.24812},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.24812},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_bf64af22b497; sha256 92bee89032dd78551203ac2ff24c281a0f7b2ebfd0a277742d5bc372b2acaf76},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.24812. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Voice provides a natural and immediate interface for AI agents. Many settings in which voice agents could be useful, including meetings, households, and collaborative work, are inherently multi-speaker. Supporting these settings introduces challenges that are largely absent from one-on-one interaction. We introduce the Multi-Speaker Interaction Benchmark (MSI-Bench) for evaluating multi-speaker voice interaction. Each test case is a short multi-party multi-turn audio scene with participant context, expected tool calls, and atomic rubrics. The benchmark targets three capability families: multi-speaker memory, multi-speaker instruction following, and multi-speaker reasoning. It comprises 1,152 test cases, evenly split between Mandarin Chinese and English (576 each). The strongest configuration on each split passes all rubrics on only 66.8% of English and 54.5% of Mandarin cases, and the strongest open-weight configuration on 34.0% and 19.3%. Failure analysis separates perception from reasoning: open-weight models are bottlenecked by the multi-speaker audio front-end, while frontier systems still fail speaker-scoped decision making on clean transcripts---and models across the board often respond when no one has addressed them. These results identify speaker-grounded perception, speaker-scoped decision making, and conversational restraint as concrete targets for future voice agents.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-22 in cs.CL).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2609.24812 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T06:48:45.063Z dalla pagina https://arxiv.org/catchup/cs.CL/2026-09-22?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).