Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Open Ant, a physical robot platform designed to facilitate the transition from simulated to real-world reinforcement learning (RL) experiments. The authors demonstrate that effective walking policies can be learned in just one hour using two RL algorithms, SARSA(λ) and Soft Actor-Critic, directly from the robot's physical interactions. Additionally, they highlight the platform's user-friendly design, which allows researchers from diverse backgrounds to quickly achieve success and easily maintain the system, making it a valuable tool for RL research.
Learning effective walking policies on a physical robot in under an hour could revolutionize how RL researchers validate their algorithms in real-world settings.
Reinforcement learning (RL) research has demonstrated success in both physical and simulated domains; however, the predominant methodology remains rooted in simulations. The predominance of simulations makes translating research to physical reality uncertain for both algorithms and researchers. We propose a physical platform that is designed to simplify the transition. In this paper, we present the Open Ant: a physical variant of the commonly used Gymnasium Ant environment, along with a simulation. We demonstrate that competent walking policies can be learned from scratch in approximately one hour directly from the physical robot's experience for two substantially different RL algorithms: SARSA($\lambda$) and Soft Actor-Critic (SAC). Separately, we show policies that were learned in simulation transfer to reality. We also examine how well the platform supports a nimble experimental ecosystem. Specifically, we observe the speed with which new users from diverse backgrounds achieve their first success with the platform, and how easily the platform can be repaired and updated when hardware issues arise. Both the hardware design and software are available as open-source on GitHub for ease of customization. In summary, we advocate for the use of the Open Ant for RL researchers who frequently use simulated environments, so they can more easily include robot experiments in their evaluations.