Search papers, labs, and topics across Lattice.
The paper introduces D-SECURE, a framework that combines content-based manipulation detection (HAMMER) with retrieval-based fact-checking (DEFAME) to improve misinformation detection in news-style posts. D-SECURE addresses the limitations of unimodal approaches by using DEFAME for broad verification and HAMMER for analyzing residual cases potentially containing fine-grained manipulations. Experiments on DGM4 and ClaimReview datasets demonstrate the complementary strengths of the integrated system, resulting in a unified and explainable misinformation detection report.
Spotting subtle multimodal misinformation requires fusing internal manipulation cues with external knowledge, as single-source methods often fail on realistic image/text fabrications.
Multimodal misinformation increasingly mixes realistic im-age edits with fluent but misleading text, producing persuasive posts that are difficult to verify. Existing systems usually rely on a single evidence source. Content-based detectors identify local inconsistencies within an image and its caption but cannot determine global factual truth. Retrieval-based fact-checkers reason over external evidence but treat inputs as coarse claims and often miss subtle visual or textual manipulations. This separation creates failure cases where internally consistent fabrications bypass manipulation detectors and fact-checkers verify claims that contain pixel-level or token-level corruption. We present D-SECURE, a framework that combines internal manipulation detection with external evidence-based reasoning for news-style posts. D-SECURE integrates the HAMMER manipulation detector with the DEFAME retrieval pipeline. DEFAME performs broad verification, and HAMMER analyses residual or uncertain cases that may contain fine-grained edits. Experiments on DGM4 and ClaimReview samples highlight the complementary strengths of both systems and motivate their fusion. We provide a unified, explainable report that incorporates manipulation cues and external evidence.
