Search papers, labs, and topics across Lattice.
This paper addresses the sim-to-real gap in deep reinforcement learning for autonomous underwater vehicle (AUV) docking by developing a high-fidelity digital twin environment using the Stonefish simulator. They accelerated the training process by implementing a multiprocessing RL framework with realistic AUV dynamics and sensor noise. Using Proximal Policy Optimization (PPO), the trained 6-DoF control policy achieved over 90% success in simulation and was successfully validated in a physical test tank, demonstrating effective sim-to-reality adaptation.
AUVs trained in simulation can successfully dock in the real world, exhibiting emergent behaviors like pitch-based braking and yaw oscillations for mechanical alignment.
Deep Reinforcement Learning (DRL) offers a robust alternative to traditional control methods for autonomous underwater docking, particularly in adapting to unpredictable environmental conditions. However, bridging the"sim-to-real"gap and managing high training latencies remain significant bottlenecks for practical deployment. This paper presents a systematic approach for autonomous docking using the Girona Autonomous Underwater Vehicle (AUV) by leveraging a high-fidelity digital twin environment. We adapted the Stonefish simulator into a multiprocessing RL framework to significantly accelerate the learning process while incorporating realistic AUV dynamics, collision models, and sensor noise. Using the Proximal Policy Optimization (PPO) algorithm, we developed a 6-DoF control policy trained in a headless environment with randomized starting positions to ensure generalized performance. Our reward structure accounts for distance, orientation, action smoothness, and adaptive collision penalties to facilitate soft docking. Experimental results demonstrate that the agent achieved a success rate of over 90% in simulation. Furthermore, successful validation in a physical test tank confirmed the efficacy of the sim-to-reality adaptation, with the DRL controller exhibiting emergent behaviors such as pitch-based braking and yaw oscillations to assist in mechanical alignment.
