Search papers, labs, and topics across Lattice.
This paper introduces a teacher-student framework for vision-based robot navigation, where a PPO-trained teacher policy using privileged 2D LiDAR data trains a student policy that relies solely on monocular depth maps. The student policy uses Depth Anything V2 for depth estimation from four RGB cameras and runs entirely onboard an NVIDIA Jetson Orin AGX. Experimental results demonstrate that the student policy outperforms the LiDAR-based teacher, especially in environments with complex 3D obstacles, achieving success rates of 82-96.5% in simulation and improved performance in real-world settings.
Vision-based robot navigation, powered by monocular depth estimation, can now outperform traditional LiDAR-based systems, particularly when navigating complex 3D environments.
Reliable obstacle avoidance in industrial settings demands 3D scene understanding, but widely used 2D LiDAR sensors perceive only a single horizontal slice of the environment, missing critical obstacles above or below the scan plane. We present a teacher-student framework for vision-based mobile robot navigation that eliminates the need for LiDAR sensors. A teacher policy trained via Proximal Policy Optimization (PPO) in NVIDIA Isaac Lab leverages privileged 2D LiDAR observations that account for the full robot footprint to learn robust navigation. The learned behavior is distilled into a student policy that relies solely on monocular depth maps predicted by a fine-tuned Depth Anything V2 model from four RGB cameras. The complete inference pipeline, comprising monocular depth estimation (MDE), policy execution, and motor control, runs entirely onboard an NVIDIA Jetson Orin AGX mounted on a DJI RoboMaster platform, requiring no external computation for inference. In simulation, the student achieves success rates of 82-96.5%, consistently outperforming the standard 2D LiDAR teacher (50-89%). In real-world experiments, the MDE-based student outperforms the 2D LiDAR teacher when navigating around obstacles with complex 3D geometries, such as overhanging structures and low-profile objects, that fall outside the single scan plane of a 2D LiDAR.
