Search papers, labs, and topics across Lattice.
This paper introduces KinematicRL, a novel framework that enhances deep reinforcement learning for social navigation by addressing the sim-to-real gap through higher-order control inputs and a robust human tracking pipeline. The authors demonstrate that using second-order control inputs significantly reduces tracking errors and improves navigation policies, while a cluster-based human tracking system using only 2D LiDAR effectively estimates pedestrian behavior without the complexity of camera-LiDAR fusion. Experimental results show that KinematicRL achieves superior kinematic performance in real-world environments, enabling reliable deployment on differential drive robots with minimal modifications.
KinematicRL bridges the sim-to-real gap in social navigation by leveraging higher-order control and a streamlined human tracking system, yielding robust real-world performance.
Deep Reinforcement Learning (DRL) has shown promise for social navigation, yet its real-world deployment remains hindered by a persistent sim-to-real gap arising from simplified first-order dynamics and context-specific human state estimation pipelines. This work presents a unified framework that addresses these limitations to produce dynamically feasible navigation policies suitable for real-world deployment. First, theoretical analysis reveals that tracking error between simulated and actual robot position decays exponentially with increased control order, motivating the use of higher-order control inputs as DRL action space. A second-order control formulation tailored to differential drive robots is developed, complemented by a stochastic iterative Linear Quadratic Regulator (iLQR) that pretrains the policy via a divergence minimization objective. Second, to avoid the added system complexity of camera-LiDAR fusion, a cluster-based human tracking pipeline using only 2D LiDAR is introduced. Human detections are associated according to both spatial proximity and velocity similarity, enabling reliable differentiation of nearby pedestrians and yielding stable velocity estimates through temporal aggregation. Third, we introduce an unbiased residual gating block to balance reaction- and memory-based behaviors while handling time-varying crowd sizes, both critical for social navigation. The resulting policy, KinematicRL, consistently improves kinematic performance and adapts to varying number of detected humans. Experiments in real-world environments demonstrate that, when combined with the proposed tracking pipeline, KinematicRL can be deployed on a real differential drive robot with minimal modifications.
