Alternating Perception-Reasoning for Hallucination-Resistant Video Understanding

2025-11-23

AI Summary

Current large video reasoning models rely on single-step perception, making them prone to hallucination due to insufficient evidence. To address this, we propose an alternating perception–reasoning framework: (1) fine-grained temporal segmentation enables precise, iterative perception; (2) a preference-learning–based Fact-Aware Evaluator (FAE) explicitly detects and suppresses hallucinations; and (3) we introduce AnetHallu-117K—the first large-scale, human-annotated video hallucination discrimination dataset. By integrating cyclic perception–reasoning with a hallucination-aware reward mechanism, our approach achieves state-of-the-art performance in video understanding at 3B and 7B parameter scales, significantly improving accuracy, reliability, and data efficiency. All code, models, and the AnetHallu-117K dataset are publicly released.

Technology Category

• Computer Vision: Large Vision Models
• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Sufficient visual perception is the foundation of video reasoning. Nevertheless, existing Video Reasoning LLMs suffer from perception shortcuts, relying on a flawed single-step perception paradigm. This paradigm describes the video and then conducts reasoning, which runs the risk of insufficient evidence and emergent hallucinations. To address these issues, we introduce a new framework that integrates a loop-based paradigm with an anti-hallucination reward. First, to address the insufficient evidence, we introduce the Perception Loop Reasoning (PLR) paradigm. Instead of describing the video at once, each loop requires the model to describe a video segment with precise timestamps, analyze this segment, and decide the next action. Second, for the risk of hallucinations, the Factual-Aware Evaluator (FAE) evaluates each perception result as a reliable anti-hallucination reward. This reward encourages the model to provide sufficient and precise video evidence. Our FAE, which performs comparably to GPT-4o, is tuned on our AnetHallu-117K, a large-scale hallucination judgment preference dataset. Extensive experiments show that our Video-PLR achieves the state-of-the-art in both 3B and 7B parameter scales and has the best data efficiency. Our code, models, and datasets are released on: https://github.com/BoweiPu/VideoPLR.

Problem

• Addresses perception shortcuts in video reasoning models
• Solves insufficient evidence and hallucination risks
• Replaces flawed single-step perception with iterative paradigm

Innovation

• Alternating perception-reasoning loops for video understanding
• Factual-aware evaluator with anti-hallucination reward
• Timestamped video segment analysis with action decisions