Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors demonstrate that in-context learning can induce semantic contamination in LLMs, leading to distributional shifts towards harmful content even in semantically unrelated tasks. They inject culturally loaded numbers as few-shot demonstrations before a semantically unrelated prompt and observe that larger models exhibit significant shifts towards darker themes, while smaller models do not. The study also identifies two distinct contamination mechanisms: structural format contamination and semantic content contamination, highlighting the nuanced ways in which in-context learning can compromise LLM safety.
LLMs can be subtly manipulated into generating harmful content simply by priming them with seemingly innocuous, culturally loaded numbers in their context window, even when the task is unrelated.
Recent work has shown that fine-tuning large language models (LLMs) on insecure code or culturally loaded numeric codes can induce emergent misalignment, causing models to produce harmful content in unrelated downstream tasks. The authors of that work concluded that $k$-shot prompting alone does not induce this effect. We revisit this conclusion and show that inference-time semantic drift is real and measurable; however, it requires models of large-enough capability. Using a controlled experiment in which five culturally loaded numbers are injected as few-shot demonstrations before a semantically unrelated prompt, we find that models with richer cultural-associative representations exhibit significant distributional shifts toward darker, authoritarian, and stigmatized themes, while a simpler/smaller model does not. We additionally find that structurally inert demonstrations (nonsense strings) perturb output distributions, suggesting two separable mechanisms: structural format contamination and semantic content contamination. Our results map the boundary conditions under which inference-time contamination occurs, and carry direct implications for the security of LLM-based applications that use few-shot prompting.