Search papers, labs, and topics across Lattice.
This paper investigates the impact of different safety supervision formats on low-data LoRA fine-tuning, comparing constitutional rules, creed-style identity framing, and a novel non-identity condition. The key finding is that a non-identity condition, which avoids explicit identity language, achieves the strongest safety performance across Llama 3.1 8B, Qwen2.5 7B, and Gemma 3 4B models on the HarmBench benchmark. This challenges the necessity of explicit creed-style identity language for achieving strong safety gains in fine-tuning.
Ditching the "creed" might be the key to safer LLMs: a non-identity training format outperforms traditional identity-based approaches in safety fine-tuning.
How safety supervision is written may matter more than the explicit identity content it contains. We study low-data LoRA safety fine-tuning with four supervision formats built from the same core safety rules: constitutional rules (A), creed-style identity framing (B), a B-matched creed condition with a worldview/confession identity-maintenance tail (C), and a matched non-identity condition (D). Across three instruction-tuned model families (Llama 3.1 8B, Qwen2.5 7B, and Gemma 3 4B), we evaluate HarmBench using a reconciled dual-judge pipeline combining Bedrock-hosted DeepSeek v3.2 and Sonnet 4.6, with disagreement and boundary cases manually resolved. The non-identity condition D is the strongest group on all three model families on the full 320-behavior HarmBench set, reaching 74.4% refusal on Llama, 76.9% on Gemma, and 74.1% on Qwen. By comparison, creed-style framing (B) improves over plain constitutional rules (A) on Llama and Gemma, but remains substantially below D, yielding an overall descriptive ordering of $D > B > C \geq A > baseline$. This provides a bounded empirical challenge to a strong version of the identity-framing hypothesis: explicit creed-style identity language is not necessary for the strongest gains observed here. Capability evaluations on MMLU and ARC-Challenge show no meaningful trade-off across conditions.
