OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
Paper · https://arxiv.org/abs/2609.39490
Recent advances have enabled unified omni-modal models in understanding audio, vision, and language. However, existing benchmarks, training data, and learning methods largely treat the modalities independently, leaving the capability of audio-visual joint reasoning poorly evaluated and insufficiently elicited. We address this gap with a benchmark, data engine, and learning method. First, we introduce OmniReasoningBench, a benchmark where both audio and visual evidence are indispensable. It comprises 1,150 multiple-choice and open-ended questions across two tasks, reasoning over video and reasoning beyond video. Second, we develop a data engine OmniQA. It automatically constructs evidence-grounded QA pairs that explicitly necessitate audio-visual joint reasoning, together with time-stamped clue chains that guide the annotation of thinking process. Besides our benchmark, this engine produces training data OmniReasoning-SFT-112K and OmniReasoning-RL-19K. Finally, we propose an on-policy self-distillation method Modality-Factored Self-Distillation (MFSD). It evaluates each sampled response under modality-specific clue contexts, disentangling the contributions of individual clues and their cross-modal interactions for token-level credit assignment. With our training data and learning method, our model OmniReasoning-30B-A3B achieves 50.0% on OmniVideoBench and 42.5% on OmniReasoningBench, improving the base model Qwen3-Omni-30B-A3B-Thinking by 12.8 and 9.3 percentage points, respectively. Moreover, it delivers substantial gains on general and long-video benchmarks, including Video-MME-v2. We hope our work offers a solid step for facilitating future research in omni-modal joint reasoning.
First seen by Oriel on .
AI summary
OmniReasoning introduces a benchmark, data engine, and training method for audio-visual joint reasoning, reporting gains over Qwen3-Omni-30B-A3B-Thinking on its benchmarks.
- OmniReasoningBench contains 1,150 multiple-choice and open-ended questions across reasoning over video and reasoning beyond video, requiring both audio and visual evidence.
- OmniQA automatically builds evidence-grounded question-answer pairs and timestamped clue chains, generating OmniReasoning-SFT-112K and OmniReasoning-RL-19K training data.
- Modality-Factored Self-Distillation evaluates responses with modality-specific clue contexts to assign token-level credit for individual clues and cross-modal interactions.
- OmniReasoning-30B-A3B scores 50.0% on OmniVideoBench and 42.5% on OmniReasoningBench, gains of 12.8 and 9.3 percentage points over the base model.
- The authors also report substantial gains on general and long-video benchmarks, including Video-MME-v2.
#audio-visual reasoning #multimodal learning #video understanding #self-distillation
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hugging Face Papers: 29, latest observation ; +1 over observed period since
Comments
Comment count reported by the source.
- Hugging Face Papers: 2, latest observation ; 0 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Papers | 2609.39490 | https://arxiv.org/abs/2609.39490 Read discussion |