Search papers, labs, and topics across Lattice.
This paper investigates the security and privacy vulnerabilities reported by developers using LLM-native Integrated Development Environments (LIDEs), analyzing 1.1 million posts from relevant subreddits to identify and categorize issues. The findings reveal that the majority of security concerns arise from system-level design choices rather than the underlying LLM technology, with developers expressing significant mistrust and resorting to external safeguards like code sandboxing. This research underscores the critical need for improved design practices in LIDEs to enhance security and privacy for users.
Developers are turning to external safeguards due to deep-seated mistrust in LIDEs, revealing systemic vulnerabilities that could compromise security and privacy.
LLM-native IDEs (Integrated Development Environments), aka LIDEs, are designed from the ground up to work with Large Language Models (LLMs). LIDEs have found remarkable success in Software Engineering (SE) tasks such as coding, debugging, and program comprehension. LIDEs are software systems, and, like any system, they can exhibit vulnerabilities. In this paper, we study the security and privacy issues that developers reported while using popular LIDEs in their development tasks. We collected 1.1M posts from 29 popular subreddits related to LIDEs. We identified 446 posts and analyzed over 6K comments to the posts that discussed security and privacy issues in almost all popular LIDEs, such as Cursor, Copilot, Codex, etc. Using a mix of qualitative and quantitative methods, we constructed a taxonomy of the reported security and privacy issues. Our results show that most issues in LIDEs stem from system-level design choices, rather than the underlying LLMs, such as user data access, unchecked autonomous actions, etc. To overcome these issues, developers frequently relied on external safeguards like code sandboxing and manual reviewing, highlighting prevalent mistrust among developers about LIDEs. We share lessons from our study to support future design of secure and privacy-aware LIDEs.
