Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study enhances the transfer of reinforcement learning policies from simulation to real-world torque-controlled robots by introducing a Real2Sim2Real pipeline that integrates trajectory matching, genetic algorithm-based parameter optimization, and domain randomization. By calibrating dynamics parameters such as friction and inertia for the 7-DOF Franka Emika Panda robot, the researchers achieved improved tracking accuracy and policy robustness in real-world applications. The results indicate that precise physical modeling is crucial for developing stable and generalizable torque-based reinforcement learning policies, facilitating smoother transitions from simulated to real environments.
Accurate physical modeling can drastically improve the stability and robustness of torque-controlled robots during Sim2Real transfers, achieving significant real-world performance gains.
Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge, particularly when dealing with low-level torque control, where even small modelling inaccuracies can lead to unstable or unsafe behaviours. In this work, we propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms by combining trajectory matching, parameter optimization via genetic algorithms, and domain randomization. Using the 7-DOF Franka Emika Panda robot, we first identify friction, inertia, and gravity compensation parameters by minimizing the error between real and simulated joint trajectories. These calibrated dynamics are then used to train a TQC-based reinforcement learning agent in simulation. The trained policy is evaluated in both Gazebo and MuJoCo environments, and finally deployed on the real robot. Our results demonstrate a significant improvement in tracking accuracy and policy robustness after parameter tuning, with smooth policy transfer from simulation to the Real-World across multiple target-reaching tasks. This work highlights the effectiveness of accurate physical modelling in enabling stable and generalizable torque-based reinforcement learning policies.