Search papers, labs, and topics across Lattice.
The paper introduces WeatherCity, a novel framework for 4D urban scene reconstruction that enables controllable multi-weather transformation, addressing the limitations of existing methods in simulating diverse weather conditions. WeatherCity employs a text-guided image editing model for flexible background editing and introduces a weather Gaussian representation with shared scene features and weather-specific decoders for multi-weather modeling, enhanced by content consistency optimization. The framework also incorporates a physics-driven model to simulate dynamic weather effects, achieving high fidelity and temporal consistency in reconstruction and editing.
Imagine training autonomous vehicles in a blizzard, then switching to a sunny day, all within the same reconstructed 4D urban scene – WeatherCity makes this a reality with unprecedented control and fidelity.
Editable high-fidelity 4D scenes are crucial for autonomous driving, as they can be applied to end-to-end training and closed-loop simulation. However, existing reconstruction methods are primarily limited to replicating observed scenes and lack the capability for diverse weather simulation. While image-level weather editing methods tend to introduce scene artifacts and offer poor controllability over the weather effects. To address these limitations, we propose WeatherCity, a novel framework for 4D urban scene reconstruction and weather editing. Specifically, we leverage a text-guided image editing model to achieve flexible editing of image weather backgrounds. To tackle the challenge of multi-weather modeling, we introduce a novel weather Gaussian representation based on shared scene features and dedicated weather-specific decoders. This representation is further enhanced with a content consistency optimization, ensuring coherent modeling across different weather conditions. Additionally, we design a physics-driven model that simulates dynamic weather effects through particles and motion patterns. Extensive experiments on multiple datasets and various scenes demonstrate that WeatherCity achieves flexible controllability, high fidelity, and temporal consistency in 4D reconstruction and weather editing. Our framework not only enables fine-grained control over weather conditions (e.g., light rain and heavy snow) but also supports object-level manipulation within the scene.
