Search papers, labs, and topics across Lattice.
This paper addresses the challenge of detecting multimodal climate disinformation by enhancing vision-language models (VLMs) with external knowledge. The authors integrate VLMs with retrieved information from reverse image searches, online fact-checks, and expert sources to provide up-to-date context for assessing the veracity of image-claim pairs. Experiments demonstrate that incorporating external knowledge significantly improves the model's ability to identify misleading climate-related content.
Combatting climate change denial online requires more than just AI pattern recognition: augmenting vision-language models with real-time fact-checking and expert knowledge significantly boosts their ability to spot disinformation.
Climate disinformation has become a major challenge in today digital world, especially with the rise of misleading images and videos shared widely on social media. These false claims are often convincing and difficult to detect, which can delay actions on climate change. While vision-language models (VLMs) have been used to identify visual disinformation, they rely only on the knowledge available at the time of training. This limits their ability to reason about recent events or updates. The main goal of this paper is to overcome that limitation by combining VLMs with external knowledge. By retrieving up-to-date information such as reverse image results, online fact-checks, and trusted expert content, the system can better assess whether an image and its claim are accurate, misleading, false, or unverifiable. This approach improves the model ability to handle real-world climate disinformation and supports efforts to protect public understanding of science in a rapidly changing information landscape.
