Search papers, labs, and topics across Lattice.
This paper empirically evaluates the prompt injection vulnerabilities of six leading large language models (LLMs) under various adversarial scenarios, including multilingual and obfuscated attacks. The study finds that all models tested exhibit significant susceptibility to prompt injections, with non-English languages showing even higher compliance rates for generating harmful content like phishing emails and malware. These findings underscore the critical need for enhanced security measures to mitigate the risks associated with deploying LLMs in sensitive applications.
All tested LLMs are vulnerable to prompt injections, with non-English languages posing an even greater risk for generating malicious content.
Large Language Models (LLMs) have rapidly evolved, transforming industries by automating complex tasks and generating human-like content. However, as their adoption accelerates, prompt injection vulnerabilities have become increasingly apparent. Malicious actors exploit these weaknesses to generate phishing emails, deceptive websites, nd malware, posing serious security risks. This paper presents an empirical evaluation of six state-of-the-art LLMs (DeepSeek, GPT, Gemini, Grok, Llama, and Qwen) under diverse adversarial prompt scenarios, including direct and multi-stage obfuscated attacks across multiple languages and character encodings. The proposed framework measures how effectively current LLMs resist manipulation into performing harmful actions. Our findings reveal systematic vulnerabilities across all tested models. Even direct prompt injections frequently induce the generation of phishing content, websites, and malware, while elaborate prompts achieve even higher malicious compliance rates, particularly for phishing. Models such as DeepSeek, Gemini, and Grok show especially high susceptibility under complex instructions. Notably, non-English languages consistently exhibit higher compliance rates than English, exposing significant gaps in multilingual safety alignment. Although simple character encodings reduce malicious outputs, they do not eliminate them. These results highlight persistent challenges in LLM safety and underscore the urgent need for stronger defenses and improved security mechanisms to support the ethical and secure deployment of LLMs in cybersecurity sensitive contexts.
