Metraipolis › archive › dc_1d2b072fc990

Multimodal Target Speaker Extraction: Towards Unified Speaker Cues Across Modalities — arXiv:2609.35613

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.35613
License (as archived)
CC0-1.0
Archived
2026-10-08T00:18:24.528Z
SHA-256 of the archived text
684449783af15eae572e4e6e39c99b356a200c64e10885183cedafbc012e20dc
Stable URL of this page
https://metraipolis.com/documento/dc_1d2b072fc990

How to cite

Multimodal Target Speaker Extraction: Towards Unified Speaker Cues Across Modalities — arXiv:2609.35613 — arXiv (en), https://arxiv.org/abs/2609.35613 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_1d2b072fc990 (Metraipolis archive dc_1d2b072fc990, sha256 684449783af15eae572e4e6e39c99b356a200c64e10885183cedafbc012e20dc)
BibTeX
@misc{dc1d2b072fc990,
  title = {Multimodal Target Speaker Extraction: Towards Unified Speaker Cues Across Modalities — arXiv:2609.35613},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.35613},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_1d2b072fc990; sha256 684449783af15eae572e4e6e39c99b356a200c64e10885183cedafbc012e20dc},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.35613. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Target Speaker Extraction (TSE) is pivotal in speech communication and human-computer interaction, enabling the isolation of a specific speaker's voice from complex acoustic environments, i.e., the cocktail party scenario. Although traditional TSE systems conditioned on enrollment speech have progressed substantially, enrollment speech as a cue has inherent limitations. Its reliability degrades when the target and interfering speakers have similar voice characteristics, when intra-speaker variability (e.g. changes in emotion or speaking style) creates a mismatch between the enrollment and target speech, or when the enrollment itself is contaminated by noise or competing speakers. This review surveys deep-learning-based TSE from the perspective of auxiliary target cues drawn from multiple modalities. We organize existing methods according to five types of information used to isolate the target speaker: audio enrollment, visual, spatial, textual/semantic, and neural cues. We also trace the evolution from discriminative estimators to variational, diffusion, flow, codec, and foundation-model-based systems and summarize representative datasets and evaluation metrics. We review the benefits and limitations of different cues and discuss challenges involving synchronization, missing or unreliable observations, data scarcity, privacy, computational cost, and real-time operation. Finally, we summarize future directions concerning adaptive cue fusion, instruction-driven extraction, realistic evaluation, and trustworthy deployment. By jointly reviewing cue design, model architecture, training objectives, datasets, and evaluation metrics, this article provides an overview of the current landscape and open problems in multimodal TSE.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.MM).
Categorie arXiv: Sound (cs.SD) ; Multimedia (cs.MM).
Pagina del lavoro: https://arxiv.org/abs/2609.35613 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T23:55:48.887Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).