Metraipolis › archive › dc_a37df960af44

RePlay: Retrieval-Based Voice Playback for Multi-Turn spoken dialogue — arXiv:2609.31588

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31588
License (as archived)
CC0-1.0
Archived
2026-10-08T04:19:22.445Z
SHA-256 of the archived text
ba06dc84fcac8832967d6f2e2070389d8c2a59390428bedf6d1af846dbed5cbb
Stable URL of this page
https://metraipolis.com/documento/dc_a37df960af44

How to cite

RePlay: Retrieval-Based Voice Playback for Multi-Turn spoken dialogue — arXiv:2609.31588 — arXiv (en), https://arxiv.org/abs/2609.31588 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_a37df960af44 (Metraipolis archive dc_a37df960af44, sha256 ba06dc84fcac8832967d6f2e2070389d8c2a59390428bedf6d1af846dbed5cbb)
BibTeX
@misc{dca37df960af44,
  title = {RePlay: Retrieval-Based Voice Playback for Multi-Turn spoken dialogue — arXiv:2609.31588},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31588},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_a37df960af44; sha256 ba06dc84fcac8832967d6f2e2070389d8c2a59390428bedf6d1af846dbed5cbb},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31588. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Many voice interaction applications require exact control over both the content and delivery of responses, typically using pre-recorded lines. Recent full-duplex models respond with low latency but cannot guarantee exact content or reproduce a specific recorded performance, while cascaded systems can be constrained to predefined responses at the cost of additional latency. We propose RePlay, a spoken dialogue system adapted from PersonaPlex that handles multi-turn conversations by retrieving and playing pre-recorded lines. Using probing, we identify the layer and frame at which the upcoming response becomes recoverable, and use this hidden state as the retrieval query. RePlay retains only the layers up to that point and replaces text and speech generation with lightweight turn-taking and retrieval heads. In simulated multi-turn interviews, RePlay reaches a median latency of 383 ms, 3 to 7 times lower than ASR-LLM cascades of comparable dialogue quality, at the cost of lower exact-line accuracy. In a user study, participants preferred RePlay in 63% of ratings versus 12% for a fast cascade with a small LLM (p = 0.008), and showed a non-significant preference (46% vs. 21%) over a slower cascade with a stronger LLM.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-28 in eess.AS).
Categorie arXiv: Audio and Speech Processing (eess.AS) ; Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2609.31588 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T03:50:06.027Z dalla pagina https://arxiv.org/catchup/eess.AS/2026-09-28?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).