Search papers, labs, and topics across Lattice.
This paper introduces RegisterBridgeMM, a register-centric framework for RGB-infrared object detection that leverages pretrained register tokens to facilitate cross-modal communication while minimizing the need for dense patch-to-patch interactions. By employing a three-stage register lifecycle鈥擜ggregate, Bridge, and Project鈥攖he framework effectively preserves modality-specific information and enhances feature calibration. The approach outperforms existing methods, achieving the highest mean Average Precision (mAP) across four benchmark datasets, demonstrating its robustness against challenges like illumination changes and cluttered scenes.
Pretrained register tokens can revolutionize RGB-IR object detection by enabling efficient cross-modal communication without sacrificing performance.
RGB-infrared (RGB-IR) object detection benefits from complementary visible and thermal cues, but effective fusion remains challenging under illumination changes, weather variation, and cluttered scenes. Existing RGB-IR fusion methods often trade expressive patch-level interaction for lighter but more constrained adaptation mechanisms. We empirically observe that pretrained register tokens contain both modality-shared and modality-specific information on paired RGB-IR inputs, suggesting that they can serve as a compact substrate for cross-modal communication. Building on this observation, we propose RegisterBridgeMM, a register-mediated fusion framework organized as a three-stage register lifecycle. Aggregate preserves per-modality register summarization inherited from pretraining; Bridge performs bidirectional register-to-patch reading with consensus-residual regulation; and Project translates the resulting register summary into spatially adaptive calibration of patch features. This register pathway avoids dense patch-to-patch cross-modal interaction while preserving the pretrained patch representation. With both backbone streams frozen, RegisterBridgeMM achieves the highest mAP50-95 among the evaluated methods on all four benchmarks: LLVIP, M3FD, DroneVehicle, and FLIR-Aligned.
