A Very Big Video Reasoning Suite

2026-02-23

AI Summary

This work addresses the lack of systematic investigation into video reasoning capabilities of current models, primarily hindered by the absence of large-scale, structured training and evaluation data. To bridge this gap, we introduce VBVR, a massive video reasoning dataset comprising 200 structured tasks and over one million video clips, organized via a task ontology–based taxonomy. We further develop VBVR-Bench, a rule-driven and verifiable evaluation framework that enables reproducible benchmarking of video reasoning. Our dataset exceeds existing resources by three orders of magnitude in scale and facilitates, for the first time, large-scale scaling experiments in video reasoning. These experiments reveal an early emergence of cross-task generalization on unseen tasks, highlighting promising avenues for future research in video understanding.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Rapid progress in video models has largely focused on visual quality, leaving their reasoning capabilities underexplored. Video reasoning grounds intelligence in spatiotemporally consistent visual environments that go beyond what text can naturally capture, enabling intuitive reasoning over spatiotemporal structure such as continuity, interaction, and causality. However, systematically studying video reasoning and its scaling behavior is hindered by the lack of large-scale training data. To address this gap, we introduce the Very Big Video Reasoning (VBVR) Dataset, an unprecedentedly large-scale resource spanning 200 curated reasoning tasks following a principled taxonomy and over one million video clips, approximately three orders of magnitude larger than existing datasets. We further present VBVR-Bench, a verifiable evaluation framework that moves beyond model-based judging by incorporating rule-based, human-aligned scorers, enabling reproducible and interpretable diagnosis of video reasoning capabilities. Leveraging the VBVR suite, we conduct one of the first large-scale scaling studies of video reasoning and observe early signs of emergent generalization to unseen reasoning tasks. Together, VBVR lays a foundation for the next stage of research in generalizable video reasoning. The data, benchmark toolkit, and models are publicly available at https://video-reason.com/ .

Problem

• video reasoning
• large-scale dataset
• spatiotemporal reasoning
• reasoning evaluation
• emergent generalization

Innovation

• video reasoning
• large-scale dataset
• emergent generalization
• verifiable evaluation
• spatiotemporal reasoning