Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the sim-to-real gap in contact-rich manipulation by training a policy to predict end-effector pose, contact state, and desired contact force direction using privileged supervision from a human-designed finite state machine controller in simulation. The key insight is that contact force *direction* is more robust to sim-to-real discrepancies than force magnitude, allowing for effective transfer. The trained policy, combined with a simple, manually tuned force magnitude, drives a force-aware admittance controller, achieving superior performance on real-world manipulation tasks compared to baselines.
Forget painstakingly tuning force magnitudes for sim-to-real transfer of contact-rich tasks; this work shows that learning just the *direction* of contact forces unlocks surprisingly robust performance.
Sim-to-real transfer for contact-rich manipulation remains challenging due to the inherent discrepancy in contact dynamics. While existing methods often rely on costly real-world data or utilize blind compliance through fixed controllers, we propose a framework that leverages expert-designed controller logic for transfer. Inspired by the success of privileged supervision in kinematic tasks, we employ a human-designed finite state machine based position/force controller in simulation to provide privileged guidance. The resulting policy is trained to predict the end-effector pose, contact state, and crucially the desired contact force direction. Unlike force magnitudes, which are highly sensitive to simulation inaccuracies, force directions encode high-level task geometry and remain robust across the sim-to-real gap. At deployment, these predictions configure a force-aware admittance controller. By combining the policy's directional intent with a constant, low-cost manually tuned force magnitude, the system generates adaptive, task-aligned compliance. This tuning is lightweight, typically requiring only a single scalar per contact state. We provide theoretical analysis for stability and robustness to disturbances. Experiments on four real-world tasks, i.e., microwave opening, peg-in-hole, whiteboard wiping, and door opening, demonstrate that our approach significantly outperforms strong baselines in both success rate and robustness. Videos are available at: https://yifei-y.github.io/project-pages/DirectionMatters/.