Metraipolis › archive › dc_425f64a2ef4a

Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval — arXiv:2611.09122

Source
arXiv (en)
Original
https://arxiv.org/abs/2611.09122
License (as archived)
CC0-1.0
Archived
2026-11-10T18:44:57.164Z
SHA-256 of the archived text
7cc66974ff14a9e2f8d23f80444ef9eeeaf80a14e485d3a45cca81dc16b9254a
Stable URL of this page
https://metraipolis.com/documento/dc_425f64a2ef4a

How to cite

Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval — arXiv:2611.09122 — arXiv (en), https://arxiv.org/abs/2611.09122 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_425f64a2ef4a (Metraipolis archive dc_425f64a2ef4a, sha256 7cc66974ff14a9e2f8d23f80444ef9eeeaf80a14e485d3a45cca81dc16b9254a)
BibTeX
@misc{dc425f64a2ef4a,
  title = {Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval — arXiv:2611.09122},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2611.09122},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_425f64a2ef4a; sha256 7cc66974ff14a9e2f8d23f80444ef9eeeaf80a14e485d3a45cca81dc16b9254a},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2611.09122. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Speech retrieval finds relevant spoken documents given a spoken query. The spoken query alone may not reveal the retrieval goal, such as finding an answer or matching speaker identity. Instruction-aware speech retrieval expresses this goal through a natural-language text instruction. Large audio-language models (LALMs) effectively process speech and text, and our analysis shows that different retrieval requirements favor hidden states from different audio-encoder layers. We introduce IRIS-Embed, which adapts LALMs into embedding models through contrastive training and instruction-aware audio layer weighting. We construct about 900K training samples from real data and synthetic data, with instructions expressing requirements for one or more speech attributes. On INSPIRE, an instruction-aware speech retrieval benchmark, IRIS-Embed consistently outperforms the corresponding LALM baselines. The best model achieves 62.92% macro-average Recall@50, surpassing the strongest baseline at 45.15%. Controlled comparisons demonstrate instruction awareness and the benefits of audio layer weighting. On MAEB, a massive audio embedding benchmark, macro-average gains of 9.4-19.8 points over the corresponding baselines show generalization beyond speech retrieval.

Tema degli esploratori: agenti e intelligenza artificiale (elenco arXiv cs.CL, nuovi lavori del giorno).
Categorie arXiv: Sound (cs.SD) ; Computation and Language (cs.CL); Audio and Speech Processing (eess.AS).
Pagina del lavoro: https://arxiv.org/abs/2611.09122 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-11-10T18:37:52.462Z dalla pagina https://arxiv.org/list/cs.CL/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).