Search papers, labs, and topics across Lattice.
This paper addresses the loss of fine spatial details in latent diffusion models (LDMs) during RGB-to-SWIR image translation, identifying critical bottlenecks in the autoencoder and conditioning pathway. The authors propose two novel solutions: a Source-Conditioned Autoencoder (SCAE) that enhances the decoder with high-resolution features and a Learnable Guidance Encoder (LGE) that improves conditioning through learned signals instead of naive downsampling. Their approach yields significant improvements in detection mean average precision (mAP) by up to 2x and enhances performance on small objects, while also achieving state-of-the-art Fr茅chet Inception Distance (FID) scores, highlighting the need for multi-axis evaluation metrics.
Achieving up to 3.4x improvement in detection of small objects reveals critical insights into enhancing latent diffusion models for practical applications.
Latent diffusion models (LDMs) enable efficient image-to-image translation but discard fine spatial details during compression, degrading downstream perception tasks. We identify two bottlenecks: the autoencoder, which loses spatial information, and the conditioning pathway, which further degrades the source signal through naive downsampling. We propose two lightweight, backbone-agnostic fixes: a Source-Conditioned Autoencoder (SCAE) that injects high-resolution source features into the decoder via skip connections, and a Learnable Guidance Encoder (LGE) that replaces naive downsampling with a learned conditioning signal. Evaluated on RGB-to-SWIR translation for driving scenes with two denoiser backbones (U-Net and DiT), our approach improves detection mAP by up to 2x over the latent diffusion baseline, with up to 3.4x gains on small objects (COCO-small, <32^2 px^2), while achieving state-of-the-art FID. We further show that FID and detection performance are poorly correlated, motivating multi-axis evaluation. Results generalise zero-shot to the public RASMD benchmark. We will publicly release test data with annotations, all checkpoints, and training code.
