Distorted or Fabricated? A Survey on Hallucination in Video LLMs

2026-04-14

AI Summary

This work addresses the susceptibility of video large language models (Vid-LLMs) to generating hallucinated content inconsistent with input videos. It presents the first systematic categorization framework for Vid-LLM hallucinations, distinguishing between dynamic distortion and content fabrication. Integrating literature review, taxonomic modeling, and causal analysis, the study synthesizes existing evaluation benchmarks, metrics, and mitigation strategies. The authors provide an in-depth investigation into the root causes of these hallucinations and propose promising future directions, including motion-aware visual encoders and integration with counterfactual learning. To support ongoing research, they also maintain a continuously updated repository of resources, fostering a structured understanding of reliability challenges in Vid-LLMs.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Despite significant progress in video-language modeling, hallucinations remain a persistent challenge in Video Large Language Models (Vid-LLMs), referring to outputs that appear plausible yet contradict the content of the input video. This survey presents a comprehensive analysis of hallucinations in Vid-LLMs and introduces a systematic taxonomy that categorizes them into two core types: dynamic distortion and content fabrication, each comprising two subtypes with representative cases. Building on this taxonomy, we review recent advances in the evaluation and mitigation of hallucinations, covering key benchmarks, metrics, and intervention strategies. We further analyze the root causes of dynamic distortion and content fabrication, which often result from limited capacity for temporal representation and insufficient visual grounding. These insights inform several promising directions for future work, including the development of motion-aware visual encoders and the integration of counterfactual learning techniques. This survey consolidates scattered progress to foster a systematic understanding of hallucinations in Vid-LLMs, laying the groundwork for building robust and reliable video-language systems. An up-to-date curated list of related works is maintained at https://github.com/hukcc/Awesome-Video-Hallucination .

Problem

• hallucination
• Video Large Language Models
• dynamic distortion
• content fabrication
• visual grounding

Innovation

• video hallucination
• dynamic distortion
• content fabrication
• temporal representation
• visual grounding