Search papers, labs, and topics across Lattice.
This paper addresses the challenges of Visible-Infrared Person Re-Identification (VI-ReID) in open-world scenarios where queries and galleries may contain homogeneous and heterogeneous modality images. The authors introduce the Visible-Infrared Modality-Incomplete Re-Identification (VIMI-ReID) task and develop the Modality Adaptive Matching Transformer (MAMT), which utilizes a Divergence Transformer Module (DTM) and a Shared Transformer Module (STM) to enhance feature extraction and matching under uncertain modality conditions. Experimental results on newly constructed SYSU-VIMI and RegDB-VIMI benchmarks show that MAMT significantly improves matching reliability and adaptability compared to existing VI-ReID methods.
Unpredictable modality combinations can drastically hinder VI-ReID performance, but the Modality Adaptive Matching Transformer (MAMT) offers a robust solution by dynamically fusing features based on modality relationships.
Visible-Infrared Person Re-Identification (VI-ReID) operates under a closed-world assumption, where queries and galleries are from heterogeneous modalities. However, in open-world scenarios, both sets are likely to contain homogeneous and heterogeneous modality images. A query may consist of visible-only, infrared-only, or mixed-modality images, while galleries present multi-modal images over long-term collection. Under these conditions, VI-ReID methods, built on a heterogeneous-modality retrieval paradigm, suffer from three trustworthiness challenges: matching conflicts due to high homogeneous-modality similarity, interference from modality uncertainty, and robustness degradation induced by unknown modality combinations. They fail to meet the requirements of trustworthy visual recognition in reliability, consistency, and dynamic adaptability. To address these challenges, we formalize the Visible-Infrared Modality-Incomplete Re-Identification (VIMI-ReID) task. We reorganize existing datasets to construct the SYSU-VIMI and RegDB-VIMI benchmarks. The unpredictable modality combinations and inherent similarity of homogeneous-modality samples in VIMI-ReID cause a significant performance drop in existing VI-ReID methods. We propose the Modality Adaptive Matching Transformer (MAMT). It employs a Divergence Transformer Module (DTM) and a Shared Transformer Module (STM) to extract modality-specific and modality-shared features, respectively. Guided by a divergence loss, the DTM enriches modality-specific features with modality-style information to enhance discriminability within the same modality. A Modality Adaptive Matching Module (MAM) dynamically fuses features according to the query-gallery modality relationship, enabling stable matching under arbitrary and uncertain modality conditions. Extensive experiments on the VIMI benchmarks demonstrate the effectiveness and adaptability of MAMT.
