Metraipolis › archive › dc_d346f28e18cf

Prompts versus Rules: Auditing and Controlling Speech Naturalness Behaviors in Voice User Simulators — arXiv:2610.11015

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.11015
License (as archived)
CC0-1.0
Archived
2026-10-09T06:04:07.982Z
SHA-256 of the archived text
117a87be17e991ef3b0e6b5edb489537932a73c1c792dd36d900adf39d126ed0
Stable URL of this page
https://metraipolis.com/documento/dc_d346f28e18cf

How to cite

Prompts versus Rules: Auditing and Controlling Speech Naturalness Behaviors in Voice User Simulators — arXiv:2610.11015 — arXiv (en), https://arxiv.org/abs/2610.11015 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d346f28e18cf (Metraipolis archive dc_d346f28e18cf, sha256 117a87be17e991ef3b0e6b5edb489537932a73c1c792dd36d900adf39d126ed0)
BibTeX
@misc{dcd346f28e18cf,
  title = {Prompts versus Rules: Auditing and Controlling Speech Naturalness Behaviors in Voice User Simulators — arXiv:2610.11015},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.11015},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d346f28e18cf; sha256 117a87be17e991ef3b0e6b5edb489537932a73c1c792dd36d900adf39d126ed0},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.11015. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

As voice agents gain more popularity commercially, the user simulators used to evaluate the deployed agents are also being developed to include more realistic, variable, and diverse speech naturalness behaviors -- disfluency, interruption and backchanneling. The quality of the user simulator directly affects the validity of agent evaluation results. However, we find that most studies so far have not examined in detail whether the intended configuration for these behaviors is realized in the simulation. In this study, we audit the realized naturalness behaviors of tau-Voice, our own LLM-based prompting approach across three models, and our rule-based injection algorithm for disfluency, interruption, and backchanneling. We find that prompting for these behaviors is unreliable and produces speech inconsistent with the instructions, placed and distributed less naturally than the instruction implies. In contrast, our rule-based, model-free algorithm produces controllable and diverse naturalness behaviors more aligned with natural speech. Our results suggest that LLMs not purpose-trained for user simulation are not sufficient on their own to represent authentic user behavior, and that linguistically informed deterministic approaches or specialized models are needed to close the gap; auditing and reporting realized naturalness behaviors, rather than configured settings, is what makes that gap visible.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Computation and Language (cs.CL).
Pagina del lavoro: https://arxiv.org/abs/2610.11015 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:47:15.107Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).