LongVQUBench: Benchmarking Long-Term Video Quality Understanding of Vision-Language Models

2026-07-01

AI Summary

Existing vision-language models lack effective evaluation for long-form video quality understanding, as prevailing benchmarks are confined to short clips and isolated distortions, overlooking temporal continuity and complex reasoning. To address this gap, this work proposes LongVQUBench—the first hierarchical evaluation benchmark tailored for long video quality understanding—comprising over 1,200 diverse real-world long videos and 1,500 hierarchically structured question-answer pairs that span local event perception, cross-event reasoning, and global quality judgment. It further introduces a novel “needle-in-a-haystack distortion” questioning paradigm to probe fine-grained recognition capabilities. Systematic evaluation of 14 state-of-the-art models reveals a marked performance decline with increasing video length and reasoning depth, exposing critical limitations in long-range temporal integration and perceptual attribution.

Abstract

The evaluation of long-term video quality understanding remains an open challenge for large vision-language models (LVLMs). Existing video quality benchmarks predominantly focus on short clips and isolated distortions, overlooking the temporal continuity, cumulative degradation, and reasoning complexity inherent in long-duration content. To address these limitations, we present LongVQUBench, a comprehensive benchmark for long-term video quality understanding. LongVQUBench contains over 1200 diverse videos spanning movies, documentaries, surveillance footage, egocentric recordings, and animated content, accompanied by 1500 multiple-choice and open-ended questions for validation and testing. To assess perceptual reasoning across different temporal scopes, we introduce three progressively complex evaluation levels: (i) local event quality understanding (LQU) for analyzing localized distortions; (ii) cross-event quality reasoning (CQR) for integrating multiple degraded events; and (iii) global quality understanding (GQU) for holistic perceptual evaluation over extended durations. Furthermore, a needle distortion question-answering (NDQA) paradigm is embedded across all three levels, where spatial or temporal artifacts are sparsely inserted to probe fine-grained detection and reasoning capabilities. Extensive experiments on 14 state-of-the-art LVLMs reveal significant performance degradation with increasing video length and reasoning depth, highlighting their limited capacity for long-range temporal integration and perceptual attribution. We envision LongVQUBench as a foundational step toward the systematic, hierarchical, and explainable evaluation of LVLMs' long-term video quality understanding.

Problem

• long-term video quality
• vision-language models
• temporal continuity
• cumulative degradation
• perceptual reasoning

Innovation

• LongVQUBench
• video quality understanding
• vision-language models
• temporal reasoning
• needle distortion QA