Video-R1: Reinforcing Video Reasoning in MLLMs

2025-03-27

AI Summary

To address the weak video reasoning capability of multimodal large language models (MLLMs), the lack of temporal modeling in existing rule-based reinforcement learning (R1) methods, and the scarcity of high-quality video reasoning data, this paper proposes the first systematic R1 paradigm. Our method introduces: (1) T-GRPO, a novel RL algorithm explicitly modeling spatiotemporal dependencies in videos; (2) an image-to-video transfer RL training paradigm that leverages abundant high-quality image reasoning data to alleviate the video annotation bottleneck; and (3) two open-source video reasoning datasets—Video-R1-COT-165k and Video-R1-260k. Evaluated on VSI-Bench (a spatial reasoning benchmark), Video-R1-7B achieves 35.8% accuracy—surpassing GPT-4o—and demonstrates significant performance gains across all major video understanding benchmarks, including VideoMMMU, MVBench, and TempCompass.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Inspired by DeepSeek-R1's success in eliciting reasoning abilities through rule-based reinforcement learning (RL), we introduce Video-R1 as the first attempt to systematically explore the R1 paradigm for eliciting video reasoning within multimodal large language models (MLLMs). However, directly applying RL training with the GRPO algorithm to video reasoning presents two primary challenges: (i) a lack of temporal modeling for video reasoning, and (ii) the scarcity of high-quality video-reasoning data. To address these issues, we first propose the T-GRPO algorithm, which encourages models to utilize temporal information in videos for reasoning. Additionally, instead of relying solely on video data, we incorporate high-quality image-reasoning data into the training process. We have constructed two datasets: Video-R1-COT-165k for SFT cold start and Video-R1-260k for RL training, both comprising image and video data. Experimental results demonstrate that Video-R1 achieves significant improvements on video reasoning benchmarks such as VideoMMMU and VSI-Bench, as well as on general video benchmarks including MVBench and TempCompass, etc. Notably, Video-R1-7B attains a 35.8% accuracy on video spatial reasoning benchmark VSI-bench, surpassing the commercial proprietary model GPT-4o. All codes, models, data are released.

Problem

• Enhancing video reasoning in MLLMs using rule-based RL
• Addressing temporal modeling gaps in video reasoning tasks
• Overcoming scarcity of high-quality video-reasoning training data

Innovation

• T-GRPO algorithm enhances temporal video reasoning
• Combines image and video data for training
• Achieves superior performance on video reasoning benchmarks