ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA

2026-07-02

AI Summary

This work addresses the challenge of accurately localizing evidence frames in long-form video question answering under a fixed input token budget, where uniform frame sampling often fails to capture question-relevant content. The authors propose a question-adaptive keyframe selection method that operates without fine-tuning the underlying multimodal large language model. The approach integrates a lightweight question-aware selector, uncertainty-guided non-maximum suppression, and a length- and difficulty-aware rethink routing mechanism to dynamically select video segments aligned with the question intent. Evaluated on Video-MME, MLVU, and LongVideoBench, the method achieves substantial gains in QA accuracy—particularly in medium- to long-duration videos—while maintaining computationally tractable overhead.

Abstract

Recent multimodal large language models (MLLMs) have substantially advanced video understanding, yet long-form video QA remains challenging under fixed input token budgets, where uniform sampling can be inefficient for evidence localization. We propose ReQuest , an uncertainty-driven, question-adaptive keyframe selection pipeline that aligns question intent with relevant video content through selective computation. ReQuest integrates (i) a lightweight question-aware selector distilled from MLLM-generated supervision, (ii) Re-thinking Routing that triggers additional inference only when the model is uncertain with a length-adaptive criterion, and (iii) uncertainty-guided adaptive non-maximum suppression that selects temporally diverse frames while adjusting spacing based on question difficulty. As a plug-andplay method, ReQuest improves long-video QA without modifying or fine-tuning the underlying MLLM. Experiments on Video-MME, MLVU, and LongVideoBench demonstrate consistent accuracy gains with competitive computational cost, with particularly strong improvements in medium and long video regimes.

Problem

• long-form video QA
• keyframe selection
• evidence localization
• multimodal large language models
• input token budget

Innovation

• question-aware frame selection
• uncertainty-driven routing
• adaptive non-maximum suppression
• long-form video QA
• multimodal large language models