Search papers, labs, and topics across Lattice.
This paper addresses the limitations of existing fact-checking systems that often produce misleading "Supports" labels by introducing SIFT, a claim-conditioned re-scoring method that evaluates the relevance of evidence spans in the context of the full claim. The authors demonstrate that SIFT significantly improves accuracy on various benchmarks, recovering up to 27.6 points in performance compared to naive decomposition methods. Additionally, the Warranted Supports Proportion (WSP) metric effectively calibrates against human-verified evidence, achieving an AUC of 0.92 and precision of 0.98, highlighting the importance of context in warrant evaluation.
SIFT reveals that context-aware re-scoring can dramatically enhance the accuracy of fact-checking systems, recovering up to 27.6 points in performance.
Fact-checking systems built on LLMs achieve high verdict accuracy on standard benchmarks, yet routinely output Supports labels whose cited evidence does not license the claim. Structured decomposition is the natural way to inspect those warrants, but rigid extraction protocols strip the full-claim context that facets need. We introduce SIFT -- claim-conditioned re-scoring of extracted evidence spans against the full claim -- paired with WSP (Warranted Supports Proportion), an automatic NLI check that the cited warrant entails the claim. We evaluate on FEVER, SciFact, 5PILS, and DP across four open-source backbones. SIFT recovers accuracy on cells where naive decomposition costs up to 27.6 points, while raising WSP above direct prompting; WSP itself calibrates against human gold evidence at AUC 0.92 and precision 0.98.
