Search papers, labs, and topics across Lattice.
This study explores the vulnerability of large language models (LLMs) to bias induction through a method called Group Relative Policy Optimization (GRPO). By applying one-shot GRPO training on a single biased example, the authors demonstrate that it is possible to induce systematic bias in LLMs, leading to stereotype-driven reasoning that generalizes across various attributes and benchmarks. The findings highlight a significant risk in current alignment strategies, showing that a single biased input can compromise model integrity and fairness.
A single biased example can completely undermine the alignment of large language models, revealing a critical flaw in their post-training safety mechanisms.
Warning: This paper contains several toxic and offensive statements. Modern large language models (LLMs) are typically aligned through large-scale post-training to ensure fair and reliable behavior. In this work, we investigate how easily such guardrails can be broken by Group Relative Policy Optimization (GRPO). We show that one-shot GRPO training on a single biased example is sufficient to induce systematic bias, with stereotype-driven reasoning generalizing across attributes, categories, and benchmarks. We further find that models differ in their susceptibility based on the initial likelihood of producing biased outputs. Our results reveal a critical vulnerability in post-training: alignment can be overridden by a single example.
