Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the optimization challenges in joint image-based (I2I) and text-based (T2I) person re-identification (ReID) by proposing a decoupled two-stage training pipeline that mitigates conflicts between the two modalities. The authors demonstrate that pre-training on I2I data enhances generalization to T2I tasks, and incorporating textual supervision during vision encoder training improves performance for both modalities. Their extensive experiments reveal that a shared representation can be effectively learned without cross-task interference, paving the way for more unified ReID systems.
Pre-training on image data can significantly boost performance in text-based person re-identification tasks, revealing a surprising synergy between modalities.
The joint optimization of image-based (I2I) and text-based (T2I) person re-identification (ReID) is hindered by modality discrepancies and conflicting training objectives, leading to suboptimal shared representations. While I2I ReID focuses on identity-level invariance across images of the same person, T2I ReID is driven by instance-specific textual descriptions tied to unique visual traits. This paper explores the fundamental difference between two ReID tasks and their optimization processes for effective training. Since I2I and T2I ReID are often studied separately, the loss functions optimized for one retrieval setting may negatively affect the representation quality required by the other. Motivated by these findings, we propose a decoupled two-stage training pipeline for learning a shared representation across image and text modalities. The pipeline is based on a single vision encoder that supports both I2I and T2I retrieval while avoiding cross-task interference during training. We provide extensive experiments across multiple configurations, varying domain mixing procedures, learning strategies, and task objectives. We observed that I2I ReID pre-training positively impacts the generalization ability to T2I data. Besides, we find that incorporating textual supervision during the vision encoder training stage enhances both I2I and T2I performance. We believe our insights provide a meaningful step toward unified ReID systems and cross-modal retrieval overall.