Search papers, labs, and topics across Lattice.
The paper introduces MarsRetrieval, a new benchmark for evaluating vision-language models on Martian geospatial data across three tasks: image-text retrieval, landform retrieval, and global geo-localization. The benchmark is designed to assess the ability of models to perform text-guided retrieval for geospatial discovery on Mars, addressing a gap in existing planetary science benchmarks. Experiments reveal that even strong foundation models struggle with domain-specific geomorphic distinctions, highlighting the necessity of domain-specific fine-tuning for effective geospatial discovery in planetary contexts.
Off-the-shelf vision-language models stumble when asked to identify Martian landforms, revealing a surprising gap in their geospatial understanding and creating a need for specialized fine-tuning.
Data-driven approaches like deep learning are rapidly advancing planetary science, particularly in Mars exploration. Despite recent progress, most existing benchmarks remain confined to closed-set supervised visual tasks and do not support text-guided retrieval for geospatial discovery. We introduce MarsRetrieval, a retrieval benchmark for evaluating vision-language models for Martian geospatial discovery. MarsRetrieval includes three tasks: (1) paired image-text retrieval, (2) landform retrieval, and (3) global geo-localization, covering multiple spatial scales and diverse geomorphic origins. We propose a unified retrieval-centric protocol to benchmark multimodal embedding architectures, including contrastive dual-tower encoders and generative vision-language models. Our evaluation shows MarsRetrieval is challenging: even strong foundation models often fail to capture domain-specific geomorphic distinctions. We further show that domain-specific fine-tuning is critical for generalizable geospatial discovery in planetary settings. Our code is available at https://github.com/ml-stat-Sustech/MarsRetrieval
