EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence

2026-06-23

AI Summary

This work addresses a critical limitation in current video question answering (VQA) evaluation, which focuses solely on answer correctness while ignoring whether models reason based on genuine temporal evidence from the video, often leading to answers disconnected from supporting visual content. To bridge this gap, the authors introduce EG-VQA, a new benchmark comprising 2,067 videos and 11,838 question-answer pairs annotated with fine-grained temporal evidence spans. They also propose EG-F1, a unified metric that jointly evaluates temporal alignment and semantic consistency between predicted answers and ground-truth evidence. Furthermore, they develop EG-Reasoner, a model that leverages explicit evidence supervision to enhance verifiable reasoning. Experiments demonstrate that EG-Reasoner achieves state-of-the-art performance among open-source models, significantly outperforming baselines on complex reasoning tasks such as counterfactuals and approaching the performance of closed-source systems, thereby revealing that mere model scaling is insufficient for robust video understanding.

Abstract

Recent advances in Video Large Language Models (Video-LLMs) have yielded promising performance on video question answering (VideoQA). Nevertheless, existing benchmarks are predominantly evaluated through answer correctness, while the grounding of predictions in relevant video evidence remains largely unexamined. This disconnect between answer generation and evidence understanding motivates the construction of the Evidence-Grounded Video Question Answering Benchmark (EG-VQA), an open-ended evaluation protocol in which each QA pair is explicitly annotated with supporting temporal evidence, thereby requiring joint reasoning and precise evidence localization. EG-VQA is comprised of 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations. To evaluate predicted evidence, Evidence-Grounded F1 (EG-F1) is introduced as a unified metric in which temporal alignment and semantic consistency against ground-truth evidence are jointly measured. Experimental evaluation reveals that even strong proprietary models struggle to accurately ground their predictions, exposing a fundamental discrepancy between answer correctness and faithful evidence localization. To bridge this gap, EG-Reasoner, an evidence-grounded reasoning model trained with explicit supervision, is proposed. State-of-the-art performance is achieved among open-source models, with results competitive against proprietary systems, particularly pronounced gains are observed on reasoning-intensive tasks such as counterfactual questions. These findings demonstrate that scaling alone is insufficient for robust video understanding and that structured evidence supervision is essential for the development of more reliable and interpretable VideoQA systems.

Problem

• Video Question Answering
• Evidence Grounding
• Temporal Localization
• Video Understanding
• Verifiable Reasoning

Innovation

• Evidence-Grounded Reasoning
• Video Question Answering
• Temporal Evidence Localization
• EG-F1 Metric
• Video-LLMs