Search papers, labs, and topics across Lattice.
This paper identifies a novel vulnerability in large language models (LLMs) termed the "pragmatic attack surface," which arises from the mismatch between implicit contextual understanding in human language and the explicit safety alignment mechanisms in LLMs. By exploiting this vulnerability, attackers can manipulate prompts to elicit unsafe outputs, achieving significantly higher success rates than traditional attack methods. The authors demonstrate that their approach consistently outperforms baseline methods across various models, highlighting the urgent need for improved safety mechanisms that account for pragmatic context.
Exploiting the mismatch between implicit human context and LLM safety alignment can lead to unprecedented attack success rates, revealing a critical vulnerability in current AI systems.
In the era of large language models (LLMs), attackers often manipulate natural language to elicit unsafe or harmful outputs, creating a new natural language attack surface unique to LLM-based systems, where attacks directly exploit explicit linguistic cues in user prompts to bypass the safety mechanism of LLMs. However, such attacks can often be mitigated by existing safety alignment algorithms. On the other hand, human language is inherently grounded in pragmatics, necessitating typical context to interpret language, e.g., world knowledge, social norms. However, such contexts are often implicit because they are not directly expressed in human language and are not sufficiently leveraged in safety alignment, creating a fundamental mismatch between human language interpretation and safety alignment approaches. In this paper, we demonstrate that this mismatch exposes vulnerabilities in LLMs. We refer to this vulnerability as the pragmatic attack surface, which can be exploited to achieve high attack success rates. The experimental results demonstrate that our proposed approach outperforms baseline attack methods across various open-source and closed-source models by a substantial margin.
