Metraipolis › archive › dc_4181b7fd618d
InteractionBench: Measuring Timing, Task Accuracy, Whisper and Volume, and Grounding in Real-Time Voice Systems — arXiv:2611.02430
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.02430
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-03T18:24:43.717Z
- SHA-256 of the archived text
20b99b6339ffacad3aa8ff3abe3b9d59133a884da45ca5e908027768740fc1f8- Stable URL of this page
https://metraipolis.com/documento/dc_4181b7fd618d
How to cite
InteractionBench: Measuring Timing, Task Accuracy, Whisper and Volume, and Grounding in Real-Time Voice Systems — arXiv:2611.02430 — arXiv (en), https://arxiv.org/abs/2611.02430 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_4181b7fd618d (Metraipolis archive dc_4181b7fd618d, sha256 20b99b6339ffacad3aa8ff3abe3b9d59133a884da45ca5e908027768740fc1f8)
BibTeX
@misc{dc4181b7fd618d,
title = {InteractionBench: Measuring Timing, Task Accuracy, Whisper and Volume, and Grounding in Real-Time Voice Systems — arXiv:2611.02430},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.02430},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_4181b7fd618d; sha256 20b99b6339ffacad3aa8ff3abe3b9d59133a884da45ca5e908027768740fc1f8},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.02430. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Most benchmarks for real-time voice systems focus on one aspect, such as turn-taking, instruction following, paralinguistic control, or conversational memory, each with its own data and set of systems. We present InteractionBench, a benchmark and evaluation harness that measures four dimensions on the same systems from the same recorded user audio: conversational timing, spoken-task accuracy, whisper and volume, and groundedness in conversations of more than twenty turns. The benchmark contains 221 items per system in thirteen scenarios. Before any system runs, we fix the expected floor-control actions for spoken tasks by rule and take the expected answers from scripted dialogues. We score whispering from the audio signal. We evaluate GPT-Live, GPT-Realtime, Gemini Live, Moshi, PersonaPlex, Freeze-Omni, and a turn-based cascade in 1,519 graded runs. Among the six full-duplex systems, timing and spoken-task accuracy are rank-correlated ($\rho = 0.83$), while the cascade has the highest spoken-task score (38/50) and the second-lowest timing score. No system corrects a user's grammar within the required window. Only Gemini Live whispers on most requests (8/10). Hosted systems pass 67-88% of groundedness probes, compared with 14-35% for open full-duplex models. Timing is measured where the client hears each system, so the timing of hosted systems includes network delay: for GPT-Live, whose server reports its own clock, removing a delay of about 1.5 s lowers its timing score from 86% to 56%. We also document where the protocol differs across systems, including the text instructions. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL); Audio and Speech Processing (eess.AS). Pagina del lavoro: https://arxiv.org/abs/2611.02430 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-03T18:10:08.050Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).