Search papers, labs, and topics across Lattice.
This paper introduces the GRASP framework, which addresses the challenges of fine-grained cross-modal understanding in drone views by implementing Region-Focused Alignment (RFA) and Semantic Perturbation Enhanced Matching (SPEM). RFA enhances object-centric alignment while minimizing background noise, and SPEM utilizes a Semantic Prototype Codebook to create semantically perturbed negatives for improved discrimination among visually similar objects. Experimental results on the GeoText-1652 benchmark and ERA dataset show that GRASP significantly improves image-text retrieval performance in aerial contexts, highlighting its effectiveness in complex visual environments.
Cross-modal focus misalignment in drone views can be mitigated, leading to a substantial boost in fine-grained image-text retrieval accuracy.
Fine-grained cross-modal understanding in drone views is essential for aerial vision-language navigation. However, the inherent wide field of view and overhead perspective of drone scenarios impose dual challenges on vision-language understanding. At the macro level, overwhelming background clutter in visual representations leads to Cross-Modal Focus Misalignment, where the model prioritizes global environmental similarities over specific object details. At the micro level, Visual Isomorphism creates ambiguity, where candidates share similar geometric structures yet differ only in subtle attributes. To address these challenges, we propose the Granularity-Aware Region Alignment and Semantic Prototype (GRASP) learning framework, enhancing discriminative capability through two synergistic strategies. Specifically, we introduce Region-Focused Alignment (RFA) to promote object-centric cross-modal alignment while suppressing background interference. Concurrently, to tackle visual isomorphism, we propose Semantic Perturbation Enhanced Matching (SPEM), which leverages a foreground-purified Semantic Prototype Codebook (SPC) to construct semantically perturbed negatives for fine-grained semantic discrimination. Extensive experiments on the GeoText-1652 benchmark and the unseen ERA dataset demonstrate that GRASP achieves competitive performance in drone-view fine-grained image-text retrieval, validating its effectiveness for cross-modal understanding in aerial scenarios. Our code implementation is available at https://github.com/UCAS-JC/GRASP.
