Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Involuntary In-Context Learning (IICL), a novel attack that leverages abstract operator framing and few-shot examples to bypass safety alignment in LLMs by forcing pattern completion. Through extensive probing of 10 OpenAI models, the authors identify key components of the attack, including the importance of semantic operator naming and example ordering. IICL achieves a 24% bypass rate on the HarmBench benchmark against GPT-5.4, demonstrating a significant vulnerability compared to direct queries.
LLM safety is more fragile than we thought: cleverly framed in-context examples can override learned refusal behaviors with alarming ease.
Safety alignment in large language models relies on behavioral training that can be overridden when sufficiently strong in-context patterns compete with learned refusal behaviors. We introduce Involuntary In-Context Learning (IICL), an attack class that uses abstract operator framing with few-shot examples to force pattern completion that overrides safety training. Through 3479 probes across 10 OpenAI models, we identify the attack's effective components through a seven-experiment ablation study. Key findings: (1)~semantic operator naming achieves 100\,\% bypass rate (50/50, $p<0.001$); (2)~the attack requires abstract framing, since identical examples in direct question-and-answer format yield 0\,\%; (3)~example ordering matters strongly (interleaved: 76\,\%, harmful-first: 6\,\%); (4)~temperature has no meaningful effect (46--56\,\% across 0.0--1.0). On the HarmBench benchmark, IICL achieves 24.0\,\% bypass $[18.6\%, 30.4\%]$ against GPT-5.4 with detailed 619-word responses, compared to 0.0\,\% for direct queries.