Metraipolis › archive › dc_b1fd48288704

OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning — arXiv:2610.12458

Source
arXiv (en)
Original
https://arxiv.org/abs/2610.12458
License (as archived)
CC0-1.0
Archived
2026-10-09T06:30:58.059Z
SHA-256 of the archived text
355626d446adb3d51ffd32f4bd0e82d7a6b1b4e8dcf5ccfd0685e23530044641
Stable URL of this page
https://metraipolis.com/documento/dc_b1fd48288704

How to cite

OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning — arXiv:2610.12458 — arXiv (en), https://arxiv.org/abs/2610.12458 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_b1fd48288704 (Metraipolis archive dc_b1fd48288704, sha256 355626d446adb3d51ffd32f4bd0e82d7a6b1b4e8dcf5ccfd0685e23530044641)
BibTeX
@misc{dcb1fd48288704,
  title = {OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning — arXiv:2610.12458},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2610.12458},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_b1fd48288704; sha256 355626d446adb3d51ffd32f4bd0e82d7a6b1b4e8dcf5ccfd0685e23530044641},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2610.12458. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that expose their capability limits. Audio--visual captioning is an ideal diagnostic task, yet current benchmarks face a coupled trade-off: whole-caption scores provide coverage without localization, local probes provide localization without coverage, and unconstrained LLM judges introduce instability. We introduce OmniCapBench (Omni-Video Caption Benchmark), a benchmark that reframes audio--visual caption evaluation as a deep-structured diagnostic framework. OmniCapBench shifts the prediction target from free-form text to sets of atomic, verifiable evaluation units across three tracks: entity references, visual shots, and audio events, enabling reliable scoring with deterministic constraint checks and localized LLM-based semantic comparisons. With 786 densely annotated videos, OmniCapBench effectively distinguishes MLLM perception errors, including temporal grounding failures, identity drift, cross-modal misalignment, and hallucinated descriptions. Evaluating frontier MLLMs reveals strong local perception but weak long-horizon audio--visual reasoning, particularly in identity drift and cross-modal misalignment, providing a fine-grained roadmap for omnimodal development.

Tema degli esploratori: statistica e scienza dei dati (elenco arXiv cs.CV, nuovi lavori del giorno).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2610.12458 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-09T05:55:07.787Z dalla pagina https://arxiv.org/list/cs.CV/new (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).