Metraipolis › archive › dc_b02f079cd68f
Heard Enough to Act: Early Tool Calling in Spoken Dialogue Systems — arXiv:2610.34653
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.34653
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-27T08:11:36.636Z
- SHA-256 of the archived text
37aafb379727eca4509a595e49822f1fda322767cc0f84dc4376495546f2e275- Stable URL of this page
https://metraipolis.com/documento/dc_b02f079cd68f
How to cite
Heard Enough to Act: Early Tool Calling in Spoken Dialogue Systems — arXiv:2610.34653 — arXiv (en), https://arxiv.org/abs/2610.34653 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b02f079cd68f (Metraipolis archive dc_b02f079cd68f, sha256 37aafb379727eca4509a595e49822f1fda322767cc0f84dc4376495546f2e275)
BibTeX
@misc{dcb02f079cd68f,
title = {Heard Enough to Act: Early Tool Calling in Spoken Dialogue Systems — arXiv:2610.34653},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.34653},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b02f079cd68f; sha256 37aafb379727eca4509a595e49822f1fda322767cc0f84dc4376495546f2e275},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.34653. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Latency-sensitive spoken agents must know not only which tool to call, but also when they have heard enough to act. Waiting for the turn to complete delays execution even when the request is already actionable. However, neither existing spoken agents nor spoken dialogue corpora define the earliest point at which a request becomes actionable. We introduce a synthetic spoken-dialogue pipeline that provides earliest-actionable-point supervision, marking when the observed speech grounds a tool and its required arguments. Through controlled task generation, speech synthesis and alignment, and gated annotation, we construct 26,694 dialogues totaling 786 hours across 48 domains, with disjoint training and test argument values. We additionally create an audio rendering of $\tau^2$-bench for external evaluation. We train an omni-modal language model on this corpus, enabling causal, chunk-level decisions and partial-transcript generation for downstream call completion. The model achieves tool-state macro-F1 scores of 0.48 and 0.47 on held-out dialogues and the external benchmark, respectively. For correct calls, the estimated mean lead over a turn-end cascade, using the same formatter, is 1.8-2.0 seconds in-domain and 1.1 seconds on $\tau^2$-bench. These results demonstrate a data-driven path toward spoken agents that act while listening. We will open-source the pipeline and the training scripts. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Computation and Language (cs.CL) ; Audio and Speech Processing (eess.AS). Pagina del lavoro: https://arxiv.org/abs/2610.34653 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-27T07:56:00.233Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).