Metraipolis › archive › dc_d27652283caa

VAmoS Part Deux: Harder, More Realistic Voice-Agent Simulation — arXiv:2609.38512

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.38512
License (as archived)
CC0-1.0
Archived
2026-10-07T12:41:54.267Z
SHA-256 of the archived text
114bb742f81b31b085b2b69629e5fec24b97886a94e258b1eaef27a3140407b9
Stable URL of this page
https://metraipolis.com/documento/dc_d27652283caa

How to cite

VAmoS Part Deux: Harder, More Realistic Voice-Agent Simulation — arXiv:2609.38512 — arXiv (en), https://arxiv.org/abs/2609.38512 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_d27652283caa (Metraipolis archive dc_d27652283caa, sha256 114bb742f81b31b085b2b69629e5fec24b97886a94e258b1eaef27a3140407b9)
BibTeX
@misc{dcd27652283caa,
  title = {VAmoS Part Deux: Harder, More Realistic Voice-Agent Simulation — arXiv:2609.38512},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.38512},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_d27652283caa; sha256 114bb742f81b31b085b2b69629e5fec24b97886a94e258b1eaef27a3140407b9},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38512. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Voice agents in production must handle several requests, background speech, and customers who lose patience. We introduce VAmoS Energy, a benchmark that combines these challenges in 100 calls about utility billing and payment assistance. Each caller makes two to four requests. The agent has sixteen tools backed by a stateful Stripe billing twin and the Apache Fineract loan engine, with account access blocked until caller verification succeeds. The tasks use public household electricity data and a policy based on Pennsylvania's residential billing rules. An LLM-as-a-verifier checks the agent's actions and spoken figures against explicit requirements. On a calibration run, it agrees with a code verifier on 99.1% of checks. Across fourteen voice stacks and three repeats per task, completion ranges from 17.3% to 44.7%. Grok Voice leads, and Gemini 3.8 Live and GPT-Live follow at about the same cost per call. Background television reduces pooled completion from 38.7% to 8.6%. The simulated caller often accepts an incorrect result because it hears the agent's words but cannot inspect its actions. These findings show why voice agents need evaluation across the whole call, including what they say, what they change, and how they handle competing speech.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.38512 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:13.639Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).