Metraipolis › archive › dc_407db73c2e92

RadarVox: Radar-Audio Multimodal Cocktail-Party Speech Separation with Speaker-Aware Cross-Modal Matching — arXiv:2609.31708

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.31708
License (as archived)
CC0-1.0
Archived
2026-10-08T00:18:23.991Z
SHA-256 of the archived text
ccae0f48d34a1e499fc301abd8bd5453e6665de4e95d5410f3f1adb75f15f964
Stable URL of this page
https://metraipolis.com/documento/dc_407db73c2e92

How to cite

RadarVox: Radar-Audio Multimodal Cocktail-Party Speech Separation with Speaker-Aware Cross-Modal Matching — arXiv:2609.31708 — arXiv (en), https://arxiv.org/abs/2609.31708 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_407db73c2e92 (Metraipolis archive dc_407db73c2e92, sha256 ccae0f48d34a1e499fc301abd8bd5453e6665de4e95d5410f3f1adb75f15f964)
BibTeX
@misc{dc407db73c2e92,
  title = {RadarVox: Radar-Audio Multimodal Cocktail-Party Speech Separation with Speaker-Aware Cross-Modal Matching — arXiv:2609.31708},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.31708},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_407db73c2e92; sha256 ccae0f48d34a1e499fc301abd8bd5453e6665de4e95d5410f3f1adb75f15f964},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.31708. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

In embodied voice interaction, cocktail-party speech perception requires both speech separation and speaker attribution across machine-generated and human speech sources. However, conventional audio-only blind source separation remains permutation ambiguous, making the correspondence between separated streams and physical speakers unclear. This paper presents RadarVox, a radar-audio multimodal benchmark for identity-aware cocktail-party speech separation. RadarVox provides acoustic mixtures and source-level radar displacement signals from loudspeaker-emitted and human speech, enabling source-aware speaker assignment. FMCW radar captures laryngeal mechanical motion, providing speaker-specific spatial-motion cues unavailable to a single-channel microphone. We inject radar-derived priors into a DPRNN separator via gated fusion and learn a speaker-aware cross-modal matcher to associate unordered speech streams with radar-observed speakers. Experiments on multi-speaker mixtures show that radar cues provide complementary benefits, achieving scale-invariant signal-to-distortion ratio (SI-SDR) values of 9.75 dB and 7.13 dB in two- and three-speaker scenarios, respectively. More importantly, the proposed method improves ordered SI-SDR by more than 13 dB over audio-only methods while achieving over 80\% speaker assignment accuracy.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-29 in cs.MM).
Categorie arXiv: Audio and Speech Processing (eess.AS) ; Multimedia (cs.MM); Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2609.31708 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T23:55:48.882Z dalla pagina https://arxiv.org/catchup/cs.MM/2026-09-29?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).