Search papers, labs, and topics across Lattice.
This paper addresses the limitations of Contrastive Reinforcement Learning (CRL) in object manipulation tasks by introducing Interaction-weighted Resampling (IWR), which accounts for the piecewise-smooth dynamics induced by object-centric interactions. The authors demonstrate that IWR enhances sample efficiency and performance in various interaction-rich environments, achieving an average improvement of 19.8% in simulation. Notably, IWR enables the first real-world goal-conditioned robot air hockey agent to significantly increase its success rate from 25% to 60%.
Interaction-weighted Resampling transforms how robots learn to manipulate objects, yielding a 19.8% boost in performance and a breakthrough in real-world goal achievement.
Contrastive Reinforcement Learning (CRL) has seen recent success in a wide variety of goal-conditioned robotics tasks by learning structured representations of the dynamics. However, despite its success in locomotion and simpler control domains, CRL often struggles in interaction-rich manipulation. We argue that a key source of this difficulty is object-centric interaction, such as contact or grasping, that induces distinct changes in the underlying dynamic modes. In this work, we formulate manipulation dynamics as a piecewise-smooth Markov process and show that interaction-induced mode changes create piecewise nonlinear reachability structures that are difficult for standard CRL energy functions to represent and plan over. Based on this analysis, we introduce Interaction-weighted Resampling (IWR). IWR performs interaction-aware resampling around phases before, during, and after interactions, encouraging the learned representation to preserve the mode boundaries that determine future reachability to capture multi-modal and piecewise nonlinear reachability. Across interaction-centric environments, including 2D dynamic control, robotic manipulation, and robot air hockey, IWR improves both sample efficiency and overall performance over prior CRL methods, with 19.8% average improvement in simulation. Finally, using a sim-to-real pipeline with policies trained by IWR, we demonstrate the first real-world goal-conditioned robot air hockey agent capable of hitting goals, improving success from 25% to 60%. Project Page: IWR-arxiv.github.io.
