Search papers, labs, and topics across Lattice.
This study explores the efficacy of modern generative models for RGB-to-IR image translation to enhance infrared vehicle detection in previously unseen UAV domains, addressing the challenge posed by the scarcity of real thermal infrared data. By training on paired RGB-IR datasets and applying these models to generate synthetic IR data from RGB images, the authors demonstrate significant improvements in detection performance using RF-DETR vehicle detectors. Notably, the use of Stable Diffusion 3.5 with ControlNet achieved a mean Average Precision (mAP) increase from 50.8 to 60.1 on Kust4K and from 25.6 to 38.4 on VTUAV, showcasing the potential of synthetic data to bridge the gap in IR data availability.
Generative RGB-to-IR translation can boost infrared vehicle detection performance by over 10 mAP points in unseen UAV domains, challenging the notion that real data is irreplaceable.
Synthetic training data is crucial for developing vision AI when real-world data is scarce, as in thermal infrared (IR) aerial vehicle detection. While abundant UAV RGB imagery motivates RGB-to-IR translation for data augmentation, unobservable thermal traits (e.g., engine heat) make learning transferable mappings challenging. This work investigates whether modern generative translators can overcome this cross-modal gap to improve infrared vehicle detection on unseen UAV target domains. Translators are trained on paired RGB-IR source datasets and applied to RGB training images from held-out target datasets to generate synthetic IR data. Evaluated methods include supervised GANs, ControlNet-based diffusion models, and foundation-model editing via LoRA. The resulting synthetic IR imagery is used to train RF-DETR vehicle detectors, which are evaluated on unseen IR target test splits across five aerial datasets, with Kust4K and VTUAV serving as target domains. Synthetic IR consistently outperforms RGB and grayscale baselines. Stable Diffusion 3.5 with ControlNet yields the best results, improving mAP from 50.8 to 60.1 on Kust4K and from 25.6 to 38.4 on VTUAV compared to models trained only on source-domain IR data. Increasing output diversity via multiple seeds (+1.1 mAP) and prompt variations (+3.3 mAP) provides additional gains on VTUAV. Although a performance gap to real target IR data remains, generative RGB-to-IR translation effectively mitigates IR data scarcity and improves cross-domain aerial vehicle detection.
