Search papers, labs, and topics across Lattice.
This paper introduces Q-Fold, a novel framework for long-video understanding that shifts the focus from isolated frames to contiguous temporal segments, allowing for a more nuanced representation of video content. By employing a query-aware approach, Q-Fold distinguishes between high-fidelity Focus Frames and less relevant contextual segments, thereby preserving critical visual information while maintaining temporal continuity. Experimental results demonstrate significant performance improvements across four long-video benchmarks, achieving up to a 9.1 percentage point gain on an ultra-long video benchmark without increasing the input budget.
Query-aware segment folding boosts long-video understanding performance by up to 9.1 percentage points without additional computational cost.
Long-video understanding remains challenging for multimodal large language models, because temporally extended videos often contain thousands of frames and are therefore expensive to process exhaustively. Existing methods usually construct compact visual inputs from long videos under a limited visual budget. However, most of them still follow a frame-centric paradigm and apply similar representations to retained content regardless of its importance. This makes it difficult to preserve both high-fidelity visual evidence and broad temporal coverage. To address this issue, we propose Q-Fold, a training-free input construction framework for long-video understanding. Instead of treating isolated frames as the basic modeling unit, Q-Fold operates on contiguous temporal segments and constructs a heterogeneous Focus--Context representation under query guidance. Query-relevant segments are preserved as high-fidelity Focus Frames, while less relevant segments are folded into chronology-preserving contextual layouts. In this way, Q-Fold preserves critical visual evidence and broad temporal coverage, while better maintaining local temporal continuity within short segments. Experiments on four long-video benchmarks with multiple Video-MLLMs show that Q-Fold consistently improves performance without increasing the input budget. Notably, it achieves gains of up to 9.1 percentage points on an ultra-long video benchmark. Code will be made publicly available.
