Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenges in multimodal fact-checking by introducing X-POSE, a benchmark of real-world, community-annotated multimodal posts, and TRENT, a novel model that employs evidence triangulation through three parallel cross-attention streams. The authors highlight the inadequacies of existing models that either rely on synthetic data or fail to capture complex interrelations between modalities. Evaluations reveal that TRENT outperforms both state-of-the-art specialized models and commercial vision-language models, demonstrating its effectiveness in real-world applications.
TRENT's innovative evidence triangulation approach significantly enhances multimodal fact-checking accuracy in real-world scenarios.
The proliferation of multimedia content on social platforms has fueled multimodal misinformation, where images are used to reinforce false claims. Consequently, Multimodal Fact-Checking (MFC) has emerged as an increasingly important research area. However, current progress is hindered by a reliance on synthetic training data and curated benchmarks that fail to capture the complexity of in-the-wild data. Furthermore, existing detection models rely on restricted intra-modality consistency or unconstrained all-to-all fusion, failing to capture nuanced relations between posts and external evidence. To address these limitations, we introduce X-POSE, a benchmark of real-world, community-annotated multimodal posts from X (formerly Twitter), augmented with full-length news articles retrieved via VLM-optimized search. Additionally, we propose TRENT, a novel MFC model that performs evidence triangulation using three parallel cross-attention streams alongside a relational fusion mechanism that explicitly models entailment and contradiction. Extensive evaluations demonstrate that TRENT consistently outperforms state-of-the-art specialized models and commercial VLMs. The code, prompt templates, and dataset are available at https://github.com/stevejpapad/evidence-triangulation