Search papers, labs, and topics across Lattice.
This paper evaluates the comprehension of negation in Multimodal Large Language Models (MLLMs) applied to Remote Sensing (RS) tasks, highlighting a critical gap in their performance when tasked with identifying false or absent information. The authors introduce RS-Neg, a benchmark designed to assess negation understanding, and reveal that existing RS MLLMs exhibit significant hallucinations and performance drops in negation tasks. To address this issue, they propose NeFo, a novel test-time learning method that enhances negation comprehension using a small fraction of unlabeled data, leading to substantial improvements and strong generalization to new tasks.
Advanced RS MLLMs struggle with negation, but a novel learning method can dramatically enhance their understanding using minimal unlabeled data.
Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in various Remote Sensing (RS) tasks. However, their ability to comprehend negation remains underexplored, limiting deployment in real-world applications where models must explicitly identify what is false or absent, e.g., emergency responders need to locate non-flooded routes for evacuation. To comprehensively study this limitation, we introduce RS-Neg, the first benchmark to evaluate negation understanding across region-level to scene-level tasks. Specifically, we design an automated data generation pipeline for RS imagery, using LLMs to synthesize diverse negation queries, and introduce a dynamic visual focus module for verification. Our evaluation reveals that advanced RS MLLMs struggle with negation, exhibiting hallucinations and substantial performance degradation. To close this gap, we propose NeFo, a novel test-time learning method that explicitly incorporates the logical role of negation into the model optimization. Remarkably, using about 5\% unlabeled test samples, NeFo significantly improves the negation understanding of models and shows strong generalization to unseen tasks. Code and data will be released upon acceptance.
Tsinghua AI