VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models
This work addresses the prevalent issue of hallucinated responses in video large language models (VideoLLMs) during open-ended video understanding, where generated answers often lack grounding in visual evidence. To mitigate this without requiring additional training, the authors propose an adaptive, training-free debiasing framework that dynamically reweights visual evidence to suppress hallucinations. The approach integrates cross-layer diagnosis of vision–text evidence flow, pre-softmax attention redistribution, masking of high-importance visual tokens, and contrastive decoding to enable fine-grained, frame-wise visual focus intervention. Experimental results demonstrate that the framework significantly improves event-level localization accuracy and temporal consistency across multiple VideoLLMs, achieving a 72.60% accuracy on the EventHallusion benchmark with LLaVA-Video-7B.