Search papers, labs, and topics across Lattice.
This paper introduces Attribute-Prompted Kernel Hashing (APKH), a novel approach for unsupervised cross-modal retrieval that leverages generalized attribute priors from vision-language foundation models to create a compact Hamming space. By employing Context-optimized Attribute Kernel Mapping (CAKM) and Kernel-Smoothed Contrastive Alignment (KSCA), APKH effectively mitigates overfitting to limited training data and enhances generalization to unseen categories. Experimental results show that APKH significantly outperforms existing state-of-the-art methods in cross-modal retrieval tasks, particularly in data-constrained environments.
Overcoming the limitations of existing unsupervised methods, APKH achieves superior cross-modal retrieval performance even with scarce data, redefining efficiency in multimodal learning.
Unsupervised cross-modal hashing enables efficient retrieval of semantically related instances across different modalities without requiring manual semantic annotation. However, existing unsupervised methods rely heavily on large-scale image-text pairs. Collecting such data can be costly, particularly in scenarios where well-aligned pairs are scarce due to privacy and specialized constraints. More critically, existing methods tend to overfit to seen training data, restricting their generalization performance on unseen categories that the constrained training data cannot cover. To address these limitations, we propose Attribute-Prompted Kernel Hashing (APKH), a novel data-efficient approach that constructs a compact, modality-aligned Hamming space driven by the generalized attribute priors of vision-language foundation models. Specifically, APKH introduces two core modules: Context-optimized Attribute Kernel Mapping (CAKM) and Kernel-Smoothed Contrastive Alignment (KSCA). CAKM formulates cross-modal alignment through hyperspherical Radial Basis Function kernel mapping, optimizing dynamic attribute kernels via prompt learning to capture modality-invariant semantics. Furthermore, KSCA extends conventional point-to-point contrastive learning by modeling limited paired data as continuous kernel distributions. This explicit smoothing of the modality gap alleviates overfitting to sparse pairwise correlations. Extensive experiments demonstrate that APKH outperforms state-of-the-art hashing methods in the challenging cross-modal retrieval tasks from seen to unseen categories under data-constrained scenarios.
