RISE-Video: Can Video Generators Decode Implicit World Rules?

2026-02-05

AI Summary

This work addresses the notable limitations of current generative video models in understanding and reasoning about implicit world rules—such as commonsense knowledge, spatial dynamics, and domain-specific constraints—and the absence of dedicated evaluation frameworks for such capabilities. To bridge this gap, we propose the first benchmark for implicit rule reasoning in text-and-image-to-video (TI2V) generation, introducing a four-dimensional evaluation protocol that assesses reasoning alignment, temporal consistency, physical plausibility, and visual quality. We further develop an automated evaluation pipeline powered by large language-multimodal models. Comprehensive experiments across 11 state-of-the-art TI2V models reveal widespread deficiencies in handling complex implicit constraints, offering critical insights and a foundational tool for advancing embodied intelligence and world simulation research.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

While generative video models have achieved remarkable visual fidelity, their capacity to internalize and reason over implicit world rules remains a critical yet under-explored frontier. To bridge this gap, we present RISE-Video, a pioneering reasoning-oriented benchmark for Text-Image-to-Video (TI2V) synthesis that shifts the evaluative focus from surface-level aesthetics to deep cognitive reasoning. RISE-Video comprises 467 meticulously human-annotated samples spanning eight rigorous categories, providing a structured testbed for probing model intelligence across diverse dimensions, ranging from commonsense and spatial dynamics to specialized subject domains. Our framework introduces a multi-dimensional evaluation protocol consisting of four metrics: \textit{Reasoning Alignment}, \textit{Temporal Consistency}, \textit{Physical Rationality}, and \textit{Visual Quality}. To further support scalable evaluation, we propose an automated pipeline leveraging Large Multimodal Models (LMMs) to emulate human-centric assessment. Extensive experiments on 11 state-of-the-art TI2V models reveal pervasive deficiencies in simulating complex scenarios under implicit constraints, offering critical insights for the advancement of future world-simulating generative models.

Problem

• implicit world rules
• video generation
• reasoning
• text-to-video
• cognitive evaluation

Innovation

• reasoning-oriented benchmark
• implicit world rules
• Text-Image-to-Video synthesis
• multimodal evaluation
• Large Multimodal Models