Metraipolis › archive › dc_425f64a2ef4a
Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval — arXiv:2611.09122
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2611.09122
- License (as archived)
- CC0-1.0
- Archived
- 2026-11-10T18:44:57.164Z
- SHA-256 of the archived text
7cc66974ff14a9e2f8d23f80444ef9eeeaf80a14e485d3a45cca81dc16b9254a- Stable URL of this page
https://metraipolis.com/documento/dc_425f64a2ef4a
How to cite
Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval — arXiv:2611.09122 — arXiv (en), https://arxiv.org/abs/2611.09122 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_425f64a2ef4a (Metraipolis archive dc_425f64a2ef4a, sha256 7cc66974ff14a9e2f8d23f80444ef9eeeaf80a14e485d3a45cca81dc16b9254a)
BibTeX
@misc{dc425f64a2ef4a,
title = {Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval — arXiv:2611.09122},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2611.09122},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_425f64a2ef4a; sha256 7cc66974ff14a9e2f8d23f80444ef9eeeaf80a14e485d3a45cca81dc16b9254a},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09122. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Speech retrieval finds relevant spoken documents given a spoken query. The spoken query alone may not reveal the retrieval goal, such as finding an answer or matching speaker identity. Instruction-aware speech retrieval expresses this goal through a natural-language text instruction. Large audio-language models (LALMs) effectively process speech and text, and our analysis shows that different retrieval requirements favor hidden states from different audio-encoder layers. We introduce IRIS-Embed, which adapts LALMs into embedding models through contrastive training and instruction-aware audio layer weighting. We construct about 900K training samples from real data and synthetic data, with instructions expressing requirements for one or more speech attributes. On INSPIRE, an instruction-aware speech retrieval benchmark, IRIS-Embed consistently outperforms the corresponding LALM baselines. The best model achieves 62.92% macro-average Recall@50, surpassing the strongest baseline at 45.15%. Controlled comparisons demonstrate instruction awareness and the benefits of audio layer weighting. On MAEB, a massive audio embedding benchmark, macro-average gains of 9.4-19.8 points over the corresponding baselines show generalization beyond speech retrieval. Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno). Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL); Audio and Speech Processing (eess.AS). Pagina del lavoro: https://arxiv.org/abs/2611.09122 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-11-10T18:37:52.462Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).