Search papers, labs, and topics across Lattice.
The paper introduces Bi-CMPStereo, a bidirectional cross-modal prompting framework, to address the challenge of modality gaps in event-frame asymmetric stereo matching. It learns aligned stereo representations by projecting each modality into both event and frame domains, effectively integrating semantic and structural features. Experiments show Bi-CMPStereo significantly improves accuracy and generalization compared to existing methods.
Bridging the gap between event and frame cameras unlocks robust 3D perception in dynamic, challenging environments by cleverly prompting each modality with the other's domain-specific cues.
Conventional frame-based cameras capture rich contextual information but suffer from limited temporal resolution and motion blur in dynamic scenes. Event cameras offer an alternative visual representation with higher dynamic range free from such limitations. The complementary characteristics of the two modalities make event-frame asymmetric stereo promising for reliable 3D perception under fast motion and challenging illumination. However, the modality gap often leads to marginalization of domain-specific cues essential for cross-modal stereo matching. In this paper, we introduce Bi-CMPStereo, a novel bidirectional cross-modal prompting framework that fully exploits semantic and structural features from both domains for robust matching. Our approach learns finely aligned stereo representations within a target canonical space and integrates complementary representations by projecting each modality into both event and frame domains. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods in accuracy and generalization.
