Search papers, labs, and topics across Lattice.
This paper introduces a framework that leverages security-oriented Metamorphic Relations (MRs) and Association Rule (AR) mining to identify and analyze vulnerabilities in code generated by large language models (LLMs). By evaluating 3,700 code snippets from five LLMs, the study reveals that a significant 68.8% of these snippets exhibit at least one security vulnerability, with hard-coded credentials and command injection being particularly prevalent. The analysis also uncovers strong co-violation patterns among vulnerabilities, indicating that certain prompt types are predictive of broader security issues, thereby highlighting the interconnected nature of security flaws in LLM-generated code.
A staggering 68.8% of LLM-generated code snippets contain security vulnerabilities, revealing a complex web of interrelated flaws that challenge traditional isolation strategies in software security.
Large language models (LLMs) frequently generate code with security vulnerabilities, yet these weaknesses are rarely isolated: they often span multiple concern areas simultaneously, reflecting the cross-cutting nature of security in software. We present a framework that combines security-oriented Metamorphic Relations (MRs) with Association Rule (AR) mining to detect vulnerabilities in LLM-generated code, uncover their co-violation structure, and trace that structure back to prompt-level risk factors. We define nine MRs covering major CWE categories, including SQL injection, XSS, command injection, path traversal, hard-coded credentials, weak cryptography, and memory-safety errors, and apply them using an LLM-based judge to 3,700 code snippets generated by five open models from the LLMSecEval benchmark. The results show that 68.8% of snippets violate at least one MR, with hard-coded credentials (79.1%) and command injection (74.4%) among the most prevalent applicable failures. AR mining reveals strong cross-cutting co-violation patterns, notably that XSS and weak cryptography co-violations predict hard-coded credentials with 82.5% confidence (lift = 3.23), along with tightly coupled clusters linking authentication, credential handling, and cryptographic weakness, as well as input-handling and memory-safety failures. We then perform prompt-level risk analysis and find that database- and authentication-related prompts are strong predictors of broad cross-cutting insecurity, while 65.5% of prompts yield consistent violation outcomes across all five models. These findings show that insecure code generation is not merely a collection of independent defects, but a structured and prompt-conditioned phenomenon, motivating cluster-aware verification and prompt-level intervention for safer LLM-assisted programming.
