Search papers, labs, and topics across Lattice.
The paper introduces MoE-ACT, a multi-task imitation learning framework for bimanual manipulation that integrates sparse Mixture-of-Experts (MoE) into the Action Chunking Transformer (ACT) encoder to mitigate task interference. By adaptively activating expert components, MoE-ACT decouples multi-task action distributions in latent space, while FiLM modulates action tokens for consistency with task instructions. Experiments in simulation and real-world dual-arm setups demonstrate that MoE-ACT outperforms vanilla ACT by 33% in success rate, showing improved robustness and generalization.
Robots can now juggle multiple tasks without dropping the ball, thanks to a new Mixture-of-Experts twist on Transformers that boosts bimanual manipulation success rates by 33%.
The ability of robots to handle multiple tasks under a unified policy is critical for deploying embodied intelligence in real-world household and industrial applications. However, out-of-distribution variation across tasks often causes severe task interference and negative transfer when training general robotic policies. To address this challenge, we propose a lightweight multi-task imitation learning framework for bimanual manipulation, termed Mixture-of-Experts-Enhanced Action Chunking Transformer (MoE-ACT), which integrates sparse Mixture-of-Experts (MoE) modules into the Transformer encoder of ACT. The MoE layer decomposes a unified task policy into independently invoked expert components. Through adaptive activation, it naturally decouples multi-task action distributions in latent space. During decoding, Feature-wise Linear Modulation (FiLM) dynamically modulates action tokens to improve consistency between action generation and task instructions. In parallel, multi-scale cross-attention enables the policy to simultaneously focus on both low-level and high-level semantic features, providing rich visual information for robotic manipulation. We further incorporate textual information, transitioning the framework from a purely vision-based model to a vision-centric, language-conditioned action generation system. Experimental validation in both simulation and a real-world dual-arm setup shows that MoE-ACT substantially improves multi-task performance. Specifically, MoE-ACT outperforms vanilla ACT by an average of 33% in success rate. These results indicate that MoE-ACT provides stronger robustness and generalization in complex multi-task bimanual manipulation environments. Our open-source project page can be found at https://j3k7.github.io/MoE-ACT/.
