Search papers, labs, and topics across Lattice.
This paper introduces a dynamic information fusion framework for Vision-Language-Action (VLA) models in robotics, addressing the limitations of static fusion methods that uniformly process all visual inputs. The framework uses a lightweight adaptive routing architecture to predict the task-relevance of multiple camera views based on the current text prompt and wrist-mounted camera observations, conditionally attenuating computations for less relevant views. Results in real-world robotic manipulation show improved inference efficiency and control performance compared to existing VLA models, demonstrating the benefits of task-aware attention.
Task-aware attention slashes compute in robot VLA models by adaptively routing visual inputs based on relevance, boosting efficiency without sacrificing control performance.
In robotics, Vision-Language-Action (VLA) models that integrate diverse multimodal signals from multi-view inputs have emerged as an effective approach. However, most prior work adopts static fusion that processes all visual inputs uniformly, which incurs unnecessary computational overhead and allows task-irrelevant background information to act as noise. Inspired by the principles of human active perception, we propose a dynamic information fusion framework designed to maximize the efficiency and robustness of VLA models. Our approach introduces a lightweight adaptive routing architecture that analyzes the current text prompt and observations from a wrist-mounted camera in real-time to predict the task-relevance of multiple camera views. By conditionally attenuating computations for views with low informational utility and selectively providing only essential visual features to the policy network, Our framework achieves computation efficiency proportional to task relevance. Furthermore, to efficiently secure large-scale annotation data for router training, we established an automated labeling pipeline utilizing Vision-Language Models (VLMs) to minimize data collection and annotation costs. Experimental results in real-world robotic manipulation scenarios demonstrate that the proposed approach achieves significant improvements in both inference efficiency and control performance compared to existing VLA models, validating the effectiveness and practicality of dynamic information fusion in resource-constrained, real-time robot control environments.
