Search papers, labs, and topics across Lattice.
The paper introduces GPT-Red, an automated red-teaming agent designed to identify novel prompt injection attacks against advanced language models. By employing a scalable self-play algorithm, GPT-Red trains on a diverse set of defender agents, achieving unprecedented robustness in adversarial training for the latest model, GPT-5.6. Key findings reveal that GPT-Red outperforms human red-teamers in attack success rates and generalizes effectively across different environments and models, marking a significant advancement in LLM safety training.
GPT-Red not only breaks previous models but also sets a new benchmark for automated red teaming, outperforming human efforts in discovering prompt injection vulnerabilities.
We introduce \textbf{GPT-Red}, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs. The goal of this model is to evaluate and improve the robustness of our production systems. To this end, we use it to adversarially train GPT-5.6, our most robust model to prompt injections to date. To create GPT-Red, we design a scalable self-play algorithm where the model is tasked with attacking a diverse population of simultaneously-trained defender agents. We train the model on realistic red-teaming environments using compute on the same scale as some of our largest RL post-training runs, making it the single-largest LLM safety training run ever documented. GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments, defender models, and harnesses. In the future, we expect that as we improve the robustness of each new GPT model, it will in turn will provide better learning signal for \textit{even stronger} red-teamer agents, thus unlocking a self-improvement flywheel.
DeepMind