Search papers, labs, and topics across Lattice.
This paper introduces EditSSC, a novel method for 3D semantic scene generation that leverages 2D Bird's Eye View (BEV) representations and a latent diffusion model to enable editing capabilities without retraining. By transforming 3D occupancy grids into multi-channel BEV images and utilizing a quantized autoencoder and UNet from Stable Diffusion, the method allows for sketch-guided generation, inpainting, and outpainting. EditSSC outperforms traditional 3D-specific architectures on the SemanticKITTI dataset, highlighting the potential of repurposing established 2D techniques for enhanced 3D scene editing and generation.
EditSSC reveals that 2D architectures can outperform complex 3D models in generating and editing semantic scenes, challenging the status quo in 3D scene generation methods.
3D semantic scene generation is crucial for autonomous driving applications, yet most methods rely on complex 3D-specific architectures such as triplane encoders and adapted diffusion networks, limiting both their simplicity and their editing capabilities. We propose EditSSC, an editing-ready method for 3D semantic scene generation using 2D Bird's Eye View (BEV) representations and off-the-shelf latent diffusion network. Our approach reshapes 3D semantic occupancy grids into multi-channel BEV images and leverages the quantized autoencoder and UNet from Stable Diffusion with minimal modifications. We perform diffusion on the latents after quantization, which enables training-free editing capabilities. By exploiting class-to-code correspondences in the codebook, our method supports sketch-guided generation, inpainting, and outpainting without any retraining. On SemanticKITTI, EditSSC outperforms existing 3D-specific baselines on unconditional generation, demonstrating that well-established 2D architectures can be effectively repurposed for 3D scene generation and editing.
