Search papers, labs, and topics across Lattice.
This paper introduces FACTWASH, an open-source tool designed to detect and flag instances of "factwashing," where AI systems obscure the verifiability of claims by omitting critical context. By leveraging a deterministic approach with named flags and evidence, FACTWASH achieves an F1 score of 0.91 on explicit negation cues, while also demonstrating that a lightweight LLM can significantly enhance recall for more complex cues like hedging and attribution. The findings reveal that 55% of problematic rewrites occur in conversational hearsay, underscoring the importance of context preservation in AI-generated content.
A staggering 55% of AI-generated conversational rewrites fail to preserve essential context, risking the integrity of information.
AI systems rewrite information constantly: conversations become stored memories, documents become answers. The rewrite can keep a claim while washing away what made it checkable, who said it, how sure they were, when it held. We call that failure factwashing, and release factwash, an open-source write-time gate that catches it deterministically, with named flags and evidence rather than an LLM judge. Building it answers a practical question: when does a cheap check suffice, and when do you need a model? What decides is whether the property has a bounded surface-cue inventory. Explicit negation cues are close to enumerable, so a word list finishes and transfers, reaching 0.91 F1 on untuned text. Hedging and attribution have open-ended realizations, so vocabulary plateaus near half recall, and a one-question LLM witness recovers +17 and +15 points of cue-detection recall at equal precision. Deployed, that witness may only lower a verdict, so it buys precision rather than coverage. We measure cue detection on 105,596 independently annotated sentences. A blind-labelled corpus of memory writes then locates the failure: 55% of bad writes in conversational hearsay, 7% in business email (p<0.001), so the first deployment question is not which detector to use but whether the failure occurs at all. On unmodified mem0 2.0.7, the gate flags 5 of 8 hedged-hearsay writes.
