Search papers, labs, and topics across Lattice.
This paper introduces WildCity, a novel city-scale multimodal dataset designed to enhance AI's ability to navigate and understand complex urban environments. By leveraging data collected from autonomous fleets over 18 extensive trajectories, the authors address the critical challenges of dynamic perception, lighting variations, and imperfect sensor data. The study establishes a baseline for urban reconstruction and outlines key challenges in developing simulation-ready urban digital twins, ultimately aiming to elevate AI's spatial reasoning capabilities to human-like levels.
WildCity reveals that AI can now tackle the complexities of urban navigation and spatial reasoning at a scale previously thought unattainable.
Humans can navigate an unfamiliar city and gradually form a coherent spatial mental map spanning tens of square kilometers. Can AI build spatial representations at a comparable scale? Although recent foundation models have advanced scene reconstruction and embodied intelligence, scaling to entire cities remains an open challenge, primarily due to the lack of city-scale data. To bridge the gap, we introduce WildCity, a real-world multimodal dataset collected by autonomous fleets traversing complex urban environments. Our dataset includes 18 trajectories, each averaging 83.7 kilometers in length, and preserves the core challenges of in-the-wild perception, e.g., dynamic objects, lighting variations, and imperfect camera poses. We further establish an urban-tailored reconstruction baseline and convert the reconstructed environments into a closed-loop simulator. Beyond the dataset and baseline, we systematically analyze the key challenges on the path to simulation-ready urban digital twins: scalability, extrapolation, and uncertainty. Ultimately, WildCity aims to catalyze progress not only in city-scale rendering, but more broadly in the pursuit of AI that can perceive, remember, and reason across space at a scale comparable to human cognition. Project page: https://han-xiangyu.github.io/Wild-City/
