Search papers, labs, and topics across Lattice.
This paper introduces the Mixture of Frames Policy (MoF), a novel approach for robotic manipulation that synchronizes action denoising across multiple coordinate frames, addressing the limitations of single-frame diffusion policies. By leveraging a column-based 6D rotation representation, MoF allows for differentiable frame transformations, enabling more efficient modeling of action distributions tailored to specific manipulation tasks. Experimental results across nine simulated tasks and two real-world scenarios demonstrate that MoF significantly outperforms traditional methods, including oracle frame selection and standard Mixture-of-Experts baselines.
Task-dependent action frames can enhance robotic manipulation performance, with MoF outperforming traditional single-frame policies in both simulation and real-world applications.
Robotic manipulation is inherently multi-frame: local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame. However, modern diffusion-based visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame. We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames. MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame. To make this possible for intermediate noisy diffusion states, we introduce a column-based 6D rotation representation within an SE(3) action parameterization that supports exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold. Across nine simulated bimanual manipulation tasks, we show that the best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines. We further evaluate MoF on two real-world bimanual mobile manipulation tasks, demonstrating that it outperforms all constituent single-frame baselines. Project homepage: https://mofpo.github.io
Stanford HAI