Search papers, labs, and topics across Lattice.
This paper conducts a systematic study of hierarchical vision-language-action (Hi-VLA) systems for robot manipulation, addressing the lack of unified design principles in existing approaches. By unifying Hi-VLA agents under an options-style control framework, the authors benchmark various design choices across different task types, revealing how model selection and interface mechanisms influence performance. The findings demonstrate that applying these distilled principles results in a significantly more capable system compared to both flat VLA control and poorly designed hierarchies, validated through simulations and real-world experiments on an ALOHA robot.
Hierarchical VLA agents can outperform traditional flat control systems by leveraging unified design principles that enhance task performance across various complexities.
Hierarchical vision-language-action (Hi-VLA) systems have emerged as a promising paradigm for complex robot manipulation, by using high-level VLM planners to decompose tasks into language subgoals executed by low-level VLA controllers. Despite recent empirical progress, there is a lack of unified design principles for these systems: existing Hi-VLA systems differ in how they choose and connect planners, controllers, mechanisms to switch between the two, and how observations and memory are represented in the planner. In this paper, we present a systematic study of Hi-VLA design for robot manipulation. We unify representative Hi-VLA agents under an options-style control framework and benchmark core design choices across short-horizon, long-horizon, and reasoning-intensive tasks. Our analysis distills practical principles for building Hi-VLA systems, showing how model choices and interface mechanisms jointly shape performance. Applying these principles yields a substantially stronger system than either flat VLA control or a naively designed hierarchy, across experiments both in simulation and on a real ALOHA robot. Overall, our results provide a foundation for building more capable, robust, and principled hierarchical VLA agents. More information and video at jiahenghu.github.io/hi-vla.
DeepMind