VidEvent: A Large Dataset for Understanding Dynamic Evolution of Events in Videos

2025-04-11

AAAI Conference on Artificial Intelligence

AI Summary

Video event understanding faces dual challenges—data scarcity and modeling complexity—due to the intricate, dynamically evolving nature of events. To address this, we introduce VidEvent: the first large-scale, finely annotated video event dataset comprising over 23,000 events. VidEvent establishes a novel paradigm for dynamic event understanding by systematically formalizing event structure along three dimensions: hierarchical composition, temporal progression, and logical dependencies. Annotations are derived from movie recap videos and rigorously validated via a multi-stage human annotation pipeline to ensure high fidelity. We further release open-source multimodal baseline models integrating CNN, Transformer, and GNN architectures, alongside a unified evaluation benchmark. Extensive experiments demonstrate state-of-the-art performance across core tasks—including event recognition, script generation, and causal reasoning—significantly advancing the field of video event understanding.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Despite the significant impact of visual events on human cognition, understanding events in videos remains a challenging task for AI due to their complex structures, semantic hierarchies, and dynamic evolution. To address this, we propose the task of video event understanding that extracts event scripts and makes predictions with these scripts from videos. To support this task, we introduce VidEvent, a large-scale dataset containing over 23,000 well-labeled events, featuring detailed event structures, broad hierarchies, and logical relations extracted from movie recap videos. The dataset was created through a meticulous annotation process, ensuring high-quality and reliable event data. We also provide comprehensive baseline models offering detailed descriptions of their architecture and performance metrics. These models serve as benchmarks for future research, facilitating comparisons and improvements. Our analysis of VidEvent and the baseline models highlights the dataset's potential to advance video event understanding and encourages the exploration of innovative algorithms and models.

Problem

• Understanding complex dynamic event evolution in videos
• Extracting event scripts for video event prediction
• Providing large-scale annotated dataset for event understanding

Innovation

• Large-scale dataset with 23,000 labeled events
• Event scripts extraction and prediction framework
• Comprehensive baseline models for benchmarking