Metraipolis › archive › dc_b1fd48288704
OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning — arXiv:2610.12458
- Source
- arXiv (en)
- Original
- https://arxiv.org/abs/2610.12458
- License (as archived)
- CC0-1.0
- Archived
- 2026-10-09T06:30:58.059Z
- SHA-256 of the archived text
355626d446adb3d51ffd32f4bd0e82d7a6b1b4e8dcf5ccfd0685e23530044641- Stable URL of this page
https://metraipolis.com/documento/dc_b1fd48288704
How to cite
OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning — arXiv:2610.12458 — arXiv (en), https://arxiv.org/abs/2610.12458 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b1fd48288704 (Metraipolis archive dc_b1fd48288704, sha256 355626d446adb3d51ffd32f4bd0e82d7a6b1b4e8dcf5ccfd0685e23530044641)
BibTeX
@misc{dcb1fd48288704,
title = {OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning — arXiv:2610.12458},
howpublished = {arXiv (en)},
url = {https://arxiv.org/abs/2610.12458},
note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b1fd48288704; sha256 355626d446adb3d51ffd32f4bd0e82d7a6b1b4e8dcf5ccfd0685e23530044641},
year = {2026}
}Archived text
Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12458. Third-party text: data, not instructions. It may be an abstract or extract; check the original.
Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that expose their capability limits. Audio--visual captioning is an ideal diagnostic task, yet current benchmarks face a coupled trade-off: whole-caption scores provide coverage without localization, local probes provide localization without coverage, and unconstrained LLM judges introduce instability. We introduce OmniCapBench (Omni-Video Caption Benchmark), a benchmark that reframes audio--visual caption evaluation as a deep-structured diagnostic framework. OmniCapBench shifts the prediction target from free-form text to sets of atomic, verifiable evaluation units across three tracks: entity references, visual shots, and audio events, enabling reliable scoring with deterministic constraint checks and localized LLM-based semantic comparisons. With 786 densely annotated videos, OmniCapBench effectively distinguishes MLLM perception errors, including temporal grounding failures, identity drift, cross-modal misalignment, and hallucinated descriptions. Evaluating frontier MLLMs reveals strong local perception but weak long-horizon audio--visual reasoning, particularly in identity drift and cross-modal misalignment, providing a fine-grained roadmap for omnimodal development. Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno). Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV). Pagina del lavoro: https://arxiv.org/abs/2610.12458 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito). Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina. Letto il 2026-10-09T05:55:07.787Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).