Search papers, labs, and topics across Lattice.
This paper investigates the assumption that visual evidence universally improves multimodal fact-checking, finding that indiscriminate use of visual data can actually decrease accuracy. To mitigate this, they introduce AMuFC, a framework with an Analyzer agent that determines the necessity of visual evidence and a Verifier agent that predicts claim veracity based on this assessment. Experiments on three datasets demonstrate that AMuFC significantly improves fact-checking performance by adaptively incorporating visual evidence.
Blindly adding images to fact-checking hurts more than it helps; this adaptive framework knows when to look.
Automated fact-checking is a crucial task not only in journalism but also across web platforms, where it supports a responsible information ecosystem and mitigates the harms of misinformation. While recent research has progressed from text-only to multimodal fact-checking, a prevailing assumption is that incorporating visual evidence universally improves performance. In this work, we challenge this assumption and show that indiscriminate use of multimodal evidence can reduce accuracy. To address this challenge, we propose AMuFC, a multimodal fact-checking framework that employs two collaborative agents with distinct roles for the adaptive use of visual evidence: An Analyzer determines whether visual evidence is necessary for claim verification, and a Verifier predicts claim veracity conditioned on both the retrieved evidence and the Analyzer's assessment. Experimental results on three datasets show that incorporating the Analyzer's assessment of visual evidence necessity into the Verifier's prediction yields substantial improvements in verification performance. In addition to all code, we release WebFC, a newly constructed dataset for evaluating fact-checking modules in a more realistic scenario, available at https://github.com/ssu-humane/AMuFC.
