Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SymmGrid, a trajectory-level augmentation framework that leverages parallelized symmetries to enhance the speed and efficiency of on-robot learning in physical robots. By modeling a Markov Decision Process (MDP) under a symmetry tree, the authors create a geometric grid structure that allows for the generation of diverse symmetric equivalences, significantly populating the replay buffer with varied experiences. The results demonstrate that SymmGrid achieves wall-clock training speed-ups of 1.37-2.17x and evaluation success rate improvements of 1.09x-1.27x across various manipulation tasks, pushing the boundaries of on-robot learning towards sub-10 minute training times.
Simple branch symmetries can accelerate on-robot learning by over 2x, bringing us closer to real-time training for complex manipulation tasks.
Deep reinforcement policy learning directly in physical robots (on-robot learning) remains bottlenecked by slow wall-clock training times. We present SymmGrid, a trajectory level augmentation framework inspired by parallelized symmetries that super-scales group transformations to significantly accelerate on-robot learning in both egocentric and exocentric visual setups. We model a Markov Decision Process (MDP) under a symmetry tree, in which state-action pairs have admissible parallelized invariant transformations that yield a geometric grid structure. The state is modelled with ego- or exocentric images and proprioception information. The latter require special treatment, in the form of homographies, to warp visual scenes in line with their corresponding spatial transformations. These parallelized transformations produce a large set of unique symmetric equivalences that populate the replay buffer with diverse and consistent experiences that speed up learning and improve performance. We present extensive training and evaluations performed directly on real robot manipulation contact tasks including peg-insertions, cable routing, and object relocations. Relative to SOTA, SymmGrid achieved wall-clock training convergence speed-ups of 1.37-2.17x, evaluation success rate improvements of 1.09x-1.27x, fastest training convergence times of 16.6, 10.9, and 79.3 minutes respectively. For trajectory wide assessments, we used normalized area under the curve (nAUC) ratios. SymmGrid achieved improvements of up to 2.59x. These results confirm that simple branch symmetries can have an outsized result due to super-scaling and bring us closer to sub-10 minute on-robot learning training in manipulation tasks suitable for arms and humanoids. The project page is available at symmgrid-robot.github.io