Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling

2026-06-23

AI Summary

This work addresses the high computational and memory costs that hinder multimodal large language models in long video understanding, as well as the inflexibility and noise sensitivity of existing keyframe sampling methods. The authors formalize video frame sampling as a quasi-Gaussian distribution problem and propose AdaQ, a training-free adaptive sampling method that dynamically adjusts the sampling interval based on the 3σ rule. AdaQ balances local and global query requirements using only a single hyperparameter. Integrated with multi-visual embeddings and the Qwen3-VL-8B model, AdaQ achieves an average performance gain of 15.8% over GPT-4o while sampling merely 64 frames, significantly outperforming current keyframe selection approaches.

Abstract

Long video understanding remains a daunting challenge for \emph{Multimodal Large Language Models} (MLLMs) due to the excessive computation and memory footprint. Thus, \emph{keyframe selection} is often adopted to mitigate this shortcoming, which however still suffers from low flexibility and high noise due to its hard sampling principle. In this paper, we define video frame selection as a problem of \emph{Quasi-Gaussian Sampling}, and propose an adaptive and training-free approach termed \textbf{\emph{AdaQ}}. Inspired by the $3$-$σ$ rule of Gaussian distribution, the objective of AdaQ is to achieve the optimal $3$-$σ$ interval for different examples, \emph{i.e.}, a smaller $3$-$σ$ interval for the local query and a larger one for the global query, thereby facilitating robust and adaptive frame sampling. To validate AdaQ, we apply it to four MLLMs with three embedding models. The extensive experimental results not only show its obvious performance gains over the default MLLMs and the SOTA keyframe selection methods, \emph{e.g.}, helping Qwen3-VL-8B outperform GPT4o by 15.8\% on average by using only 64 frames, but also confirm its superior robustness and high efficiency for long-video understanding, \emph{e.g.}, \textbf{only 1 hyper-parameter} needs to be set. \textbf{Our code project} is given at \href{https://github.com/Zkayovo-xmu/AdaQ}{https://github.com/Zkayovo-xmu/AdaQ}.

Problem

• Long video understanding
• Multimodal Large Language Models
• Keyframe selection
• Adaptive sampling
• Computational efficiency

Innovation

• Quasi-Gaussian Sampling
• Adaptive Frame Selection
• Multimodal Large Language Models
• Long Video Understanding
• Training-Free Sampling