Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces DiffPlace, a diffusion-based framework for generating place-controllable street view images from text, BEV maps, and object bounding boxes, specifically addressing the challenge of generating background-consistent urban scenes. DiffPlace employs a place-ID controller, using linear projection, a perceiver transformer, and contrastive learning to map place-ID embeddings into a CLIP space, enabling control over background consistency while allowing foreground variations. Experiments demonstrate that DiffPlace achieves superior generation quality and improves visual place recognition performance when used for data augmentation compared to existing methods.
Achieve place-controllable street view generation with a novel place-ID controller that ensures background consistency while allowing flexible foreground manipulation.
Generative models have advanced significantly in realistic image synthesis, with diffusion models excelling in quality and stability. Recent multi-view diffusion models improve 3D-aware street view generation, but they struggle to produce place-aware and background-consistent urban scenes from text, BEV maps, and object bounding boxes. This limits their effectiveness in generating realistic samples for place recognition tasks. To address these challenges, we propose DiffPlace, a novel framework that introduces a place-ID controller to enable place-controllable multi-view image generation. The place-ID controller employs linear projection, perceiver transformer, and contrastive learning to map place-ID embeddings into a fixed CLIP space, allowing the model to synthesize images with consistent background buildings while flexibly modifying foreground objects and weather conditions. Extensive experiments, including quantitative comparisons and augmented training evaluations, demonstrate that DiffPlace outperforms existing methods in both generation quality and training support for visual place recognition. Our results highlight the potential of generative models in enhancing scene-level and place-aware synthesis, providing a valuable approach for improving place recognition in autonomous driving