Search papers, labs, and topics across Lattice.
This paper systematically evaluates the capabilities and limitations of AI systems in generating secure code by introducing a three-level framework that assesses models' understanding of secure coding principles, their implementation of these principles in code generation, and the gaps between knowledge and execution. The study finds that while a strong understanding of secure coding principles correlates with improved outcomes in functional correctness and security, significant gaps persist where models fail to translate this knowledge into effective code. These insights highlight critical areas for improvement and suggest targeted strategies for enhancing secure code generation practices in AI systems.
Despite recognizing secure coding principles, AI models still struggle to implement them effectively, revealing critical gaps in code generation capabilities.
The increasing use of AI systems for code generation raises a central security question: what can today's models and coding agents actually do to produce secure code, where do they still fail, and what would move the field forward? Existing work has explored prompting, fine-tuning, reinforcement learning, and agentic workflows for secure code generation, but the field still lacks a systematic understanding of how these techniques improve security and why substantial failures persist. In this SoK, we systematize the progress, pitfalls, and paths forward for AI secure code generation. We introduce a three-level framework that measures models' natural-language understanding of secure coding principles, their code-level actuation of those principles during generation, and the knowledge--actuation gaps between the two. We instantiate this framework across models and coding agents on benchmarks covering both isolated function-level security and full web-application security. Our results show that secure-coding-principle understanding is a statistically strong predictor of code-level outcomes, including functional correctness, security, and joint functional-security correctness. Yet substantial knowledge--actuation gaps remain: models can recognize relevant security principles but still fail to translate them into secure and functional code. These findings offer a principle-centered account of where AI secure code generation stands today and identify concrete paths forward through principle-guided generation, evaluation, benchmarking, and agentic workflows.
