Search papers, labs, and topics across Lattice.
This paper introduces a red teaming framework designed to systematically identify vulnerabilities in large language models (LLMs) by employing a multi-role architecture consisting of target, attacker, and jury models. Through a case study, the framework effectively highlights issues of unfaithfulness in LLM outputs, demonstrating that adversarial prompts can increase attack success rates by up to 7.9% in question-answering tasks. The findings reveal that architectural design choices are more critical than parameter scaling in ensuring model safety, while also exposing challenges in automating adversarial prompt generation across different languages.
Exploitative adversarial prompts can boost attack success rates in LLMs by nearly 8%, revealing critical vulnerabilities in model reliability.
Large language models (LLMs) have demonstrated remarkable performance across natural language processing tasks, yet their deployment in high-stakes applications raises critical concerns regarding reliability, safety, and trustworthiness. In this paper, we present a red teaming framework that systematically uncovers vulnerabilities in LLM outputs. Our approach employs a novel multi-role architecture comprising target, attacker, and jury models. The attackers generate increasingly effective adversarial prompts while the jury rigorously evaluates response accuracy and consistency across tasks. In a case study, our strategy proved particularly effective at exposing unfaithfulness in LLM responses. Exploitative adversarial prompts increased the attack success rate by up to 7.9% in question-answering tasks, revealing weaknesses in reliability. The approach identifies how structural constraints in summarization can shape vulnerability patterns, with format limitations yielding measurable gains in faithfulness, and shows that architectural design choices typically outweigh parameter scaling in determining model safety. The framework's key strength is its adaptability across evaluation tasks, from English question-answering to Arabic summarization, enabling comprehensive comparison of model vulnerabilities. While it excels at comparing cross-model and cross-linguistic vulnerabilities, it faces challenges in fully automating adversarial prompt generation across languages. Our experiments also reveal limitations in detecting subtle forms of unfaithfulness that do not manifest as explicit factual contradictions, particularly across linguistic contexts. Overall, this architecture provides both actionable insights into current LLM vulnerabilities and a scalable methodology for ongoing safety evaluation as models evolve.
