Search papers, labs, and topics across Lattice.
This paper introduces SidewalkBench, a novel benchmark for evaluating visual navigation models specifically in urban sidewalk environments, addressing the challenges posed by complex layouts and dynamic pedestrian behaviors. Utilizing NVIDIA Isaac Sim, the benchmark features a diverse array of high-fidelity environments and incorporates reactive pedestrian animations to facilitate standardized assessments. The evaluation of nine existing models reveals that pedestrian interaction and long-horizon navigation are significant weaknesses, suggesting that enhanced training with synthetic data could improve performance in these areas.
Pedestrian interactions and long-horizon navigation are critical pain points for visual navigation models, with SidewalkBench revealing that scaling training with synthetic data could be the key to overcoming these challenges.
Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While recent visual navigation models offer a promising solution, the lack of a unified benchmark hinders quantitative and reproducible evaluation. To bridge this gap, we propose SidewalkBench, a comprehensive benchmark designed for visual navigation on urban sidewalks. Built upon NVIDIA Isaac Sim, SidewalkBench brings GPU-accelerated simulation of diverse, high-fidelity sidewalk environments, including both procedurally generated and real-world scanned scenes. We further populate the scenes with rich, reactive event-based pedestrian behaviors and flexible, efficient animation, enabling standardized model evaluation under realistic real-world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution.
