Metraipolis › archive › dc_5e2d47ca70ef

HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding — arXiv:2608.29120

Source
arXiv (en)
Original
https://arxiv.org/abs/2608.29120
License (as archived)
CC0-1.0
Archived
2026-10-17T07:47:30.100Z
SHA-256 of the archived text
7614305eee19c7cc337231404a23ad7ea9db7c4c2a2d08ddfe1f249d47d003be
Stable URL of this page
https://metraipolis.com/documento/dc_5e2d47ca70ef

How to cite

HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding — arXiv:2608.29120 — arXiv (en), https://arxiv.org/abs/2608.29120 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_5e2d47ca70ef (Metraipolis archive dc_5e2d47ca70ef, sha256 7614305eee19c7cc337231404a23ad7ea9db7c4c2a2d08ddfe1f249d47d003be)
BibTeX
@misc{dc5e2d47ca70ef,
  title = {HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding — arXiv:2608.29120},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2608.29120},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_5e2d47ca70ef; sha256 7614305eee19c7cc337231404a23ad7ea9db7c4c2a2d08ddfe1f249d47d003be},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2608.29120. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speech Language Models (SLMs) are increasingly deployed in multi-speaker environments, yet their ability to attribute speech to the correct speaker and reason over speaker identities remains unclear. Hence, we introduce HEAR, a conceptually hierarchical benchmark diagnosing the foundational capabilities of speaker-attributed reasoning, comprising 2.4K human-verified samples from 887 diverse multi-party audio clips. Evaluating 20 leading SLMs on HEAR reveals they struggle with these foundational tasks, often relying on semantic priors rather than actual vocal cues. To address this, we present A2R, a 30B model optimized on Counterfactual Audio with Speaker-level Hard negatives (CASH), a dataset designed to guide the model to prioritize acoustic vocal cues over linguistic signals. A2R achieves strong performance on HEAR and exhibits zero-shot generalization to diverse multi-speaker downstream tasks, demonstrating that learned speaker attribution unlocks the model's latent capacity for speaker-aware reasoning. All resources are available at this https URL

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-01 in cs.AI).
Categorie arXiv: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2608.29120 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-17T07:28:26.719Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-09-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).