Search papers, labs, and topics across Lattice.
This paper introduces Goggles, a pretrained module that modifies the finetuning gradient of language models to instill a specific epistemic frame, effectively addressing the issue of Negation Neglect where models fail to recognize fictional claims. By applying this method, the authors demonstrate that models can identify fictional content with 91% accuracy after training on non-annotated documents, while maintaining performance on tasks like GPQA and TruthfulQA. This approach not only enhances the model's ability to discern the nature of the information it processes but also allows for the application of different epistemic frames, thereby enabling more nuanced interactions with potentially misaligned data.
Goggles can transform how language models interpret information, achieving a staggering 91% accuracy in identifying fictional claims without altering the underlying data.
Finetuning a language model on documents that are explicitly annotated as fictional results in a model that still actually believes the documents'core claims, an effect known as Negation Neglect. In our evaluations, models trained on documents prefixed and suffixed with such annotations correctly identify the relevant claims as fictional only about 9% of the time. To address this, we introduce Goggles, a learned module that intervenes on the finetuning gradient rather than the data. During supervised finetuning, a Goggles module edits the gradients an LLM LoRA receives, imparting a chosen epistemic frame (the stance the model takes toward the nature of what it reads) to whatever the documents teach. A Goggles instance is trained once for a given base model, frame, and LoRA configuration, then applied frozen to documents it was never trained on. Trained through Goggles on those same documents, now carrying no fictional annotation, the model flags the content as fictional roughly 91% of the time, while preserving capability (GPQA and TruthfulQA match or exceed baseline). The same architecture supports other frames: a Goggles instance can be trained to treat documents as"part of an AI safety evaluation by Redwood Research"rather than simply as fiction. The imparted frame persists under continued finetuning that pushes back toward the claim, where prior interventions revert. Goggles suggests a path toward training language models on known-misaligned data without absorbing the behaviors that data demonstrates.
