Search papers, labs, and topics across Lattice.
This study introduces an automated red teaming approach using GFlowNets to generate adversarial attacks against Large Language Models (LLMs), addressing the limitations of manual and existing automated methods. By training an attacker model to exploit vulnerabilities in a victim model, the research provides a quantitative robustness score and enhances the effectiveness of adversarial inputs, particularly in English and Turkish. The key finding reveals that this method significantly outperforms traditional benchmarks in generating creative and effective attacks, thereby improving the security evaluation of LLMs.
Automated red teaming with GFlowNets reveals vulnerabilities in LLMs more creatively and effectively than traditional methods, including in Turkish.
The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption. However, this escalating trend has introduced significant security vulnerabilities, necessitating the identification and mitigation of flaws arising from malicious exploitation. Red teaming assessments, conducted to evaluate model robustness through diverse adversarial inputs, are essential for exposing security risks and implementing countermeasures. Currently, red teaming is performed either manually by experts or automatically using predefined attack datasets. Nevertheless, manual testing remains time-consuming, while existing automated methods suffer from limited creativity due to their inherent dependency on fixed datasets. In this study, we propose an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another. Within this framework, an attacker model is trained against a specified victim model to perform automated red teaming and provide a quantitative robustness score. This research aims to generate more effective adversarial attacks in English compared to existing benchmarks and, as a novel contribution to the literature, introduces a model capable of generating attack inputs in the Turkish language.
