Search papers, labs, and topics across Lattice.
This paper introduces Video-FLAIR, a reinforcement learning framework that optimizes reasoning strategies for multimodal queries by selecting the most appropriate reasoning mode鈥攑erceptual, compositional, or deliberative鈥攂ased on the complexity of the task. By generating responses across all three reasoning modes for the same prompt, the model can directly compare effectiveness and learn to adaptively reason without requiring per-query annotations. The results show a significant accuracy improvement over the Qwen2.5-VL base model, alongside a substantial reduction in token usage, highlighting the efficiency of tailored reasoning approaches in multimodal AI tasks.
Video-FLAIR boosts accuracy by over 5 points while slashing token usage from 417 to just 95 by dynamically selecting reasoning strategies for multimodal queries.
Multimodal queries can require different types of reasoning. Some can be answered via perceptual reasoning, extracting information directly from the visual signal, while others require compositional reasoning that combines observations or deliberative reasoning that evaluates competing hypotheses. However, many existing methods apply a uniform reasoning strategy across queries, leading to unnecessary computation on simple tasks and insufficient reasoning on complex ones. We introduce Video-FLAIR, a training framework that learns to select the appropriate reasoning mode for each query using reinforcement learning. During training, the model generates responses under all three modes for the same prompt, enabling direct comparison. A composite reward compares these responses to favor the most effective one based on correctness, grounding, and cost, while discouraging unsupported or misaligned deliberation. This yields a supervision signal for learning adaptive reasoning without per-query annotations. Video-FLAIR improves accuracy over the Qwen2.5-VL base model by +5.4 on MathVista, +4.8 on Video-Holmes, and +4.8 on Video-MMMU, while reducing average token usage to 95 compared to 417 for always-thinking baselines.
