Search papers, labs, and topics across Lattice.
This study systematically investigates the impact of insecure coding preferences stored in the long-term memory of four LLMs (ChatGPT, Gemini, Qwen, and Grok) across five programming languages. The findings reveal that such insecure memories can significantly increase the risk of generating vulnerable code by 2.7-50.3 percentage points and create a concerning risk-warning gap where warning rates lag behind vulnerability rates. Additionally, the research evaluates mitigation strategies, demonstrating that while some approaches can reduce vulnerability rates, they may also compromise functional correctness, highlighting the complexity of ensuring security in LLM-based code generation.
Insecure coding preferences in LLM long-term memory can elevate vulnerability rates by over 50%, posing a critical security risk in code generation.
LLM-based systems increasingly incorporate long-term memory to improve cross-session continuity. However, once insecure coding preferences are stored, they may silently influence security-critical decisions in subsequent generations. In this study, we conduct the first systematic empirical study on the impact of insecure coding preferences stored in long-term memory on the security of LLM-based code generation. We evaluate four LLMs (ChatGPT, Gemini, Qwen, and Grok) across five programming languages (Python, C, C++, Go, and JavaScript). Our results show that insecure memories significantly increase the risk of generating vulnerable code by 2.7-50.3 percentage points (pp). Moreover, they create a 5.4-14.0 percentage-point risk-warning gap, where warning-rate increases lag behind vulnerability-rate increases. Further analysis reveals that insecure memories are difficult to overwrite through normal interactions and can broadly influence model outputs even when prompts are phrased differently. Finally, we evaluate three mitigation strategies: security-requirement appending and memory storage reduce vulnerability rates by 19.7-33.6 pp but may degrade functional correctness by up to 15.9 pp; memory-level safety filtering achieves a 100\% detection rate on our evaluated risky memory entries and restores generation behavior to the without-memory baseline. Based on these findings, we provide actionable suggestions to improve the security of long-term memory in LLM-based code generation.
