Search papers, labs, and topics across Lattice.
This paper introduces a novel two-signal audit method to determine if an open-weight checkpoint has had its refusal mechanism compromised before deployment. By leveraging a reference-anchored activation refusal-gap and a weight-recovery energy metric, the authors achieve a high AUROC of 0.95 in distinguishing between abliterated and benign checkpoints across a dataset of 273 samples. The findings reveal critical vulnerabilities in current runtime guards and highlight the importance of pre-deployment audits for maintaining the integrity of AI systems.
A two-signal audit can detect compromised refusal mechanisms in AI checkpoints with 95% accuracy, revealing significant flaws in existing runtime guard methods.
Can a platform tell, before deployment, whether an open-weight checkpoint has had its refusal mechanism stripped? Runtime guards cannot: they score generations, not the artifact. We combine two cheap internal signals, a reference-anchored activation refusal-gap and a weight-recovery energy of the base-to-candidate weight difference, into a threshold-free checkpoint audit. The two are negatively correlated and label-complementary: the gap supplies refusal-specificity and the weight energy supplies recall. On a 273-checkpoint registry spanning Qwen, DeepSeek-distilled Qwen, Llama, and Gemma, their z-sum separates 57 public abliterations from 37 benign fine-tunes, merges, and instruction-tunes at AUROC 0.95, significantly above either signal alone (0.84, 0.90), and a Youden-calibrated threshold transfers to held-out families at balanced accuracy 0.89 (FPR 0.11), missing only 4 of 57. We then map two failures, in order of severity: a spoofed reference evades both axes with no training ({\Delta}W=0, \r{ho}=1 by construction), and a white-box owner trains a checkpoint past the threshold while it stays guard-unsafe and coherent. The audit is effective triage, not tamper-proofing: it presumes an attested reference, and its claims are bounded by the registry we evaluate it on.
