Search papers, labs, and topics across Lattice.
This paper addresses the challenge of long video understanding in Multimodal Large Language Models (MLLMs) by introducing an adaptive frame selection method called AdaQ, which utilizes Quasi-Gaussian Sampling to optimize keyframe selection. By applying the $3$-$蟽$ rule, AdaQ dynamically adjusts the sampling interval based on the context of the query, leading to improved robustness and efficiency. Experimental results demonstrate that AdaQ significantly enhances performance, allowing models like Qwen3-VL-8B to outperform GPT4o by 15.8% while using only 64 frames and requiring minimal hyperparameter tuning.
AdaQ enables MLLMs to achieve superior long video understanding with just 64 frames, outperforming state-of-the-art methods by a striking margin.
Long video understanding remains a daunting challenge for \emph{Multimodal Large Language Models} (MLLMs) due to the excessive computation and memory footprint. Thus, \emph{keyframe selection} is often adopted to mitigate this shortcoming, which however still suffers from low flexibility and high noise due to its hard sampling principle. In this paper, we define video frame selection as a problem of \emph{Quasi-Gaussian Sampling}, and propose an adaptive and training-free approach termed \textbf{\emph{AdaQ}}. Inspired by the $3$-$蟽$ rule of Gaussian distribution, the objective of AdaQ is to achieve the optimal $3$-$蟽$ interval for different examples, \emph{i.e.}, a smaller $3$-$蟽$ interval for the local query and a larger one for the global query, thereby facilitating robust and adaptive frame sampling. To validate AdaQ, we apply it to four MLLMs with three embedding models. The extensive experimental results not only show its obvious performance gains over the default MLLMs and the SOTA keyframe selection methods, \emph{e.g.}, helping Qwen3-VL-8B outperform GPT4o by 15.8\% on average by using only 64 frames, but also confirm its superior robustness and high efficiency for long-video understanding, \emph{e.g.}, \textbf{only 1 hyper-parameter} needs to be set. \textbf{Our code project} is given at \href{https://github.com/Zkayovo-xmu/AdaQ}{https://github.com/Zkayovo-xmu/AdaQ}.
