Metraipolis › archive › dc_45ca2eefede0

Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-Visual LLMs — arXiv:2610.21588

Source: arXiv (en)
Original: https://arxiv.org/abs/2610.21588
License (as archived): CC0-1.0
Archived: 2026-10-16T10:35:22.539Z
SHA-256 of the archived text: 3d4ff1efb5f95854069882325e2625e78203f0624cfe4fac1a960e4fac94bd02
Stable URL of this page: https://metraipolis.com/documento/dc_45ca2eefede0

How to cite

Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-Visual LLMs — arXiv:2610.21588 — arXiv (en), https://arxiv.org/abs/2610.21588 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_45ca2eefede0 (Metraipolis archive dc_45ca2eefede0, sha256 3d4ff1efb5f95854069882325e2625e78203f0624cfe4fac1a960e4fac94bd02)

BibTeX

@misc{dc45ca2eefede0,
  title = {Hear, Localize, and Reason: Spatially Aware Scene Understanding for Audio-Visual LLMs — arXiv:2610.21588},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.21588},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_45ca2eefede0; sha256 3d4ff1efb5f95854069882325e2625e78203f0624cfe4fac1a960e4fac94bd02},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.21588. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Audio-visual large language models (AV-LLMs) have made strong progress in multimodal understanding, yet most treat audio as monaural semantic content, limiting their ability to reason about where sounds originate and how they relate to the visual scene. Recent spatial audio-visual studies address parts of this problem, but often focus on specific abilities, such as spatial correspondence or direction and distance reasoning. In this work, we introduce HLR-AVSceneQA, a comprehensive benchmark with a hear-localize-reason hierarchy that evaluates whether models can recognize what is heard, localize sound sources in egocentric and allocentric views, and use these localized sources to reason about how they relate to the visual scene. We further introduce HLR-LLM, which augments a strong audio-visual foundation model with a binaural spatial audio branch alongside its monaural semantic audio pathway. We train HLR-LLM with a curriculum that incorporates final-stage chain-of-thought supervision for multi-step spatial reasoning. Experiments show that existing AV-LLMs remain limited in spatial grounding and relational reasoning, while HLR-LLM substantially improves spatial audio-visual scene understanding without compromising semantic audio-visual perception.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Sound (cs.SD).
Pagina del lavoro: https://arxiv.org/abs/2610.21588 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-16T08:52:02.670Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).