Search papers, labs, and topics across Lattice.
This paper formalizes the "abstract sim2real" problem, where simulators lack key task details, using state abstraction from RL. They show that grounding abstract dynamics with state history enables successful policy transfer. They introduce a method to correct abstract simulator dynamics using real-world task data, facilitating sim2sim and sim2real transfer.
Transferring policies from coarse, incomplete simulators to the real world is possible by correcting the simulator's dynamics with real-world data.
In recent years, reinforcement learning (RL) has shown remarkable success in robotics when a fast and accurate simulator is available for a given task. When using RL and simulation, more simulator realism is generally beneficial but becomes harder to obtain as robots are deployed in increasingly complex and widescale domains. In such settings, simulators will likely fail to model all relevant details of a given target task and this observation motivates the study of sim2real with simulators that leave out key task details. In this paper, we formalize and study the abstract sim2real problem: given an abstract simulator that models a target task at a coarse level of abstraction, how can we train a policy with RL in the abstract simulator and successfully transfer it to the real-world? Our first contribution is to formalize this problem using the language of state abstraction from the RL literature. This framing shows that an abstract simulator can be grounded to match the target task if the grounded abstract dynamics take the history of states into account. Based on the formalism, we then introduce a method that uses real-world task data to correct the dynamics of the abstract simulator. We then show that this method enables successful policy transfer both in sim2sim and sim2real evaluation.
