Search papers, labs, and topics across Lattice.
This paper implements an Adversarial Prompting Framework (APF) to assess the safety of Generative AI models against adversarial prompt attacks (APAs). By generating structured adversarial prompts at varying sophistication levels, the framework quantitatively evaluates model resilience and identifies vulnerabilities across different attack vectors. The findings reveal that encoded prompts are particularly effective, achieving the highest success rates in circumventing safety mechanisms, highlighting critical gaps in current AI safety measures.
Encoded adversarial prompts can bypass AI safety mechanisms with alarming success rates, revealing significant vulnerabilities in current models.
Artificial Intelligence (AI), especially Generative AI (GenAI), adoption has increased in industries significantly in recent years. However, the use of these models may also expose systems to new forms of cyberattacks by different malicious actors -- adversarial prompt attack (APA) being one of the most prominent examples of such threats. This paper presents the implementation of an Adversarial Prompting Framework (APF) for a comprehensive assessment of AI safety. The framework systematically evaluates the resilience of the AI model through the generation of structured adversarial prompts at multiple sophistication levels, from direct harmful requests to advanced encoding-based attacks. Our implementation demonstrates the practical application of this methodology in enterprise environments, providing automated testing capabilities with quantitative security assessment metrics. The results indicate significant variations in the model vulnerabilities across different attack vectors, with encoded prompts presenting the highest success rates in bypassing safety mechanisms.
Microsoft Research