Metraipolis › archive › dc_2fdce0ab80e0
Voices as Handles: Reasoning about Speaker Identity with Frozen Text LLMs — arXiv:2609.38501
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2609.38501
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-07T16:18:40.919Z
- SHA-256 of the archived text
7c63368260fa5b83a85ea4808b0d8c1c361356eebe863d02a17e61ad2fb57ddf- Stable URL of this page
https://metraipolis.com/documento/dc_2fdce0ab80e0
How to cite
Voices as Handles: Reasoning about Speaker Identity with Frozen Text LLMs — arXiv:2609.38501 — arXiv (en), https://arxiv.org/abs/2609.38501 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_2fdce0ab80e0 (Metraipolis archive dc_2fdce0ab80e0, sha256 7c63368260fa5b83a85ea4808b0d8c1c361356eebe863d02a17e61ad2fb57ddf)
BibTeX
@misc{dc2fdce0ab80e0,
title = {Voices as Handles: Reasoning about Speaker Identity with Frozen Text LLMs — arXiv:2609.38501},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2609.38501},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_2fdce0ab80e0; sha256 7c63368260fa5b83a85ea4808b0d8c1c361356eebe863d02a17e61ad2fb57ddf},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.38501. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multi-user voice agents must track who said what across dialogue sessions. Text LLMs are attractive backbones for such agents, but transcripts alone do not expose acoustic speaker identity, leaving the model without a persistent reference for linking information to speakers across sessions. We address this gap by introducing Speaker Handles, soft-token representations that expose acoustic speaker identity to a frozen text LLM for cross-session speaker-dependent reasoning. A three-stage curriculum trains a lightweight projector, with fewer than 0.1% of the backbone's parameters, to map speaker embeddings into these handles. Establishing whether the resulting handles truly support cross-session speaker-dependent reasoning is challenging with existing benchmarks because textual cues can partially reveal fact ownership. We therefore present SpeakerBind, a controlled shared-agent benchmark in which overlapping facts across users require correct cross-session speaker attribution. Speaker Handles achieve 97.40-98.36% accuracy on VoxCeleb1 and 70.40% on SpeakerBind, close to the 71.88% topline. These results show that the proposed Speaker Handles provide an efficient way to integrate acoustic speaker identity into frozen text LLMs for speaker-content reasoning. Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in eess.AS). Categorie arXiv: Audio and Speech Processing (eess.AS) ; Sound (cs.SD). Pagina del lavoro: https://arxiv.org/abs/2609.38501 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-07T15:51:53.220Z dalla pagina https://arxiv.org/catchup/eess.AS/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).