Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To prevent undesired behavioral leakage and capability loss caused by standard inoculation prompting during supervised fine-tuning, the authors develop Stratified Inoculation Prompting (SIP). This approach tackles common regimes where desired and undesired behaviors heavily co-occur, rendering complete data filtering prohibitive due to extreme clean-data scarcity. SIP consistently suppresses emergent misalignment across harmful-advice benchmarks while preserving significantly more target capabilities than standard inoculation, and introduces password-locking to restrict elicitation strictly to explicit triggers.
Standard inoculation prompting often cripples capabilities and fails to contain bad behaviors, but diversifying prompt contexts across just a tiny fraction of clean data effectively cordons off emergent misalignment.
Supervised fine-tuning can teach language models undesired behaviours alongside desired ones. Inoculation prompting (IP) aims to limit unwanted generalisation by requesting the undesired behaviour during training and removing the request at inference. However, undesired behaviour can still appear under unrelated prompts. IP can also hinder learning of the desired behaviour. We address these limitations in settings where both behaviours co-occur in most training examples, so filtering out examples with undesired behaviour leaves only a small clean subset. We introduce stratified inoculation prompting (SIP). SIP leverages a small clean subset to demonstrate that desired behaviour should persist without the undesired one across different contexts. SIP oversamples these clean examples under diverse non-eliciting prompts while inoculating the rest. SIP substantially reduces expression of undesired behaviour while preserving more of the desired behaviour than IP. These gains persist even when we extend IP to oversample the same clean subset at the same rate as SIP. Moreover, SIP yields lower emergent misalignment rates in all harmful-advice setups we tested. SIP can be further extended to limit the undesired behaviour even under prompts that explicitly request it. We introduce backdoor dilution, which weakens expression under the inoculation prompt, and password-locked inoculation, which concentrates elicitation on a designated password. Taken together, our findings show that changing the training contexts for a small clean subset can significantly improve selective generalisation.