Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenges of transferring knowledge from RGB models to infrared (IR) imagery through a novel framework called FreqKD, which leverages the spectral structure of the RGB-IR modality gap. By analyzing feature divergence across spatial frequencies, the authors implement a frequency-decoupled knowledge distillation approach that applies asymmetric supervision to low and high-frequency components differently. The results demonstrate that FreqKD significantly improves performance on the KAIST multispectral pedestrian detection task, achieving a mAP50 of 64.1, surpassing the DINOv2 baseline by 2.4 points and showing effective transferability across datasets and tasks.
High-frequency divergence in cross-modal knowledge distillation is 2.4 times greater than low-frequency divergence, leading to a novel framework that dramatically boosts infrared object detection performance.
Transfer learning from large-scale RGB foundation models to infrared (IR) imagery through knowledge distillation (KD) remains challenging due to fundamental differences in image formation physics. We investigate the spectral structure of the RGB--IR modality gap and observe that feature divergence is not uniform across spatial frequencies: low-frequency components (shape, layout) show greater cross-modal alignment than high-frequency components (texture, fine edges), which reflect modality-specific characteristics. Based on this analysis, we propose FreqKD, a frequency-decoupled distillation framework that applies asymmetric supervision adapted to each band's cross-modal consistency. The method employs strict mean squared error (MSE) on the low-frequency band to preserve shared structural information and a relaxed log-MSE loss (weighted at 0.1) on the high-frequency band to provide edge guidance while tolerating texture differences. Spectral divergence analysis on 500 paired samples shows that high-frequency divergence exceeds low-frequency divergence by a factor of 2.4x on average across all analysed transformer layers. On KAIST multispectral pedestrian detection, FreqKD achieves 64.1 mAP50, improving 2.4 points over the DINOv2 baseline. The learned representation transfers across datasets (FLIR ADAS, +2.1 mAP50), tasks (MFNet segmentation, +1.85 mean intersection-over-union), and architectures (ResNet-50, +1.0 mAP50). Code is available at: https://anonymous.4open.science/r/freq_decoupled_kd-5E5A