Metraipolis › archive › dc_040032647951

OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning — arXiv:2609.39490

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.39490
License (as archived)
CC0-1.0
Archived
2026-10-07T12:45:26.650Z
SHA-256 of the archived text
74d160ef00113cf3ae8a0a504f7c7fd81598cc9312b1fe568ea3c6a11117f631
Stable URL of this page
https://metraipolis.com/documento/dc_040032647951

How to cite

OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning — arXiv:2609.39490 — arXiv (en), https://arxiv.org/abs/2609.39490 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_040032647951 (Metraipolis archive dc_040032647951, sha256 74d160ef00113cf3ae8a0a504f7c7fd81598cc9312b1fe568ea3c6a11117f631)
BibTeX
@misc{dc040032647951,
  title = {OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning — arXiv:2609.39490},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.39490},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_040032647951; sha256 74d160ef00113cf3ae8a0a504f7c7fd81598cc9312b1fe568ea3c6a11117f631},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.39490. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Recent advances have enabled unified omni-modal models in understanding audio, vision, and language. However, existing benchmarks, training data, and learning methods largely treat the modalities independently, leaving the capability of audio-visual joint reasoning poorly evaluated and insufficiently elicited. We address this gap with a benchmark, data engine, and learning method. First, we introduce OmniReasoningBench, a benchmark where both audio and visual evidence are indispensable. It comprises 1,150 multiple-choice and open-ended questions across two tasks, reasoning over video and reasoning beyond video. Second, we develop a data engine OmniQA. It automatically constructs evidence-grounded QA pairs that explicitly necessitate audio-visual joint reasoning, together with time-stamped clue chains that guide the annotation of thinking process. Besides our benchmark, this engine produces training data OmniReasoning-SFT-112K and OmniReasoning-RL-19K. Finally, we propose an on-policy self-distillation method Modality-Factored Self-Distillation (MFSD). It evaluates each sampled response under modality-specific clue contexts, disentangling the contributions of individual clues and their cross-modal interactions for token-level credit assignment. With our training data and learning method, our model OmniReasoning-30B-A3B achieves 50.0% on OmniVideoBench and 42.5% on OmniReasoningBench, improving the base model Qwen3-Omni-30B-A3B-Thinking by 12.8 and 9.3 percentage points, respectively. Moreover, it delivers substantial gains on general and long-video benchmarks, including Video-MME-v2. We hope our work offers a solid step for facilitating future research in omni-modal joint reasoning.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-10-01 in cs.AI).
Categorie arXiv: Computer Vision and Pattern Recognition (cs.CV) ; Artificial Intelligence (cs.AI).
Pagina del lavoro: https://arxiv.org/abs/2609.39490 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-07T12:14:14.092Z dalla pagina https://arxiv.org/catchup/cs.AI/2026-10-01?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).