Search papers, labs, and topics across Lattice.
This study investigates the impact of Google's Gemini AI tool on software security by conducting a quantitative programming study with 159 developers. Participants were assigned a security-related programming task with either no AI assistance, the free version of Gemini, or the paid version. The results showed that while Gemini usage did not significantly improve code security, developers' programming experience did, suggesting AI tools cannot fully replace human expertise in secure coding.
Despite the hype, Google's Gemini doesn't significantly improve code security, highlighting the continued importance of developer experience in producing secure software.
The ongoing shortage of skilled developers, particularly in security-critical software development, has led organizations to increasingly adopt AI-powered development tools to boost productivity and reduce reliance on limited human expertise. These tools, often based on large language models, aim to automate routine tasks and make secure software development more accessible and efficient. However, it remains unclear how developers' general programming and security-specific experience, and the type of AI tool used (free vs. paid) affect the security of the resulting software. Therefore, we conducted a quantitative programming study with software developers (n=159) exploring the impact of Google's AI tool Gemini on code security. Participants were assigned a security-related programming task using either no AI tools, the free version, or the paid version of Gemini. While we did not observe significant differences between using Gemini in terms of secure software development, programming experience significantly improved code security and cannot be fully substituted by Gemini.
