Search papers, labs, and topics across Lattice.
This paper introduces Kinematics-supervised explicit routing (KinRT), a novel approach that enhances expert routing in mixture-of-experts (MoE) augmented visual-language agents (VLAs) by utilizing kinematic clustering of action trajectories. By shifting from implicit to explicit routing based on kinematic archetypes, KinRT effectively improves expert dispatching during inference, relying solely on visual-language observations. Experimental results show that KinRT outperforms existing methods by over 23% on the RoboTwin benchmark and 20% on a newly developed DIYRobot platform, highlighting its effectiveness in diverse manipulation tasks.
KinRT achieves over 23% improvement in expert routing accuracy by leveraging kinematic archetypes, transforming how MoE systems can operate without kinematic signals at inference.
While MoE augments VLA via expert specialization, router suffers from ineffective expert routing owing to the kinematic heterogeneity of actions across manipulation tasks and, even worse, the unavailability of the kinematic signals at inference time. In this work, we first observe that most semantically distinct manipulation tasks reduce to multiple kinematic archetypes. Motivated by this finding, we propose Kinematics-supervised explicit routing (KinRT), a new paradigm that shifts from implicit, observation-driven expert routing to explicit, kinematics-guided expert dispatching. Specifically, we perform kinematic clustering on action trajectories into multiple kinematically coherent groups, whose IDs serve as ground truth to supervise the training of the router; at inference time, the router dispatches experts only using visual-language observations, without any reliance on action kinematics. KinRT actually introduces an asymmetric bridging mechanism that distills the task kinematics from the action space in training into the observation space at inference. In addition, to assess KinRT's cross-platform generalization, we build an economical, Do-It-Yourself robot (DIYRobot) platform from scratch using 3D-print technology ($<$ 2,000USD). Extensive experiments demonstrate KinRT's superiority over both dense and MoE-featured VLAs by more than 23.26% on RoboTwin benchmark and 20.27% on our introduced DIYRobot platform. Our code and DIYRobot platform will be open-sourced.
