Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates prompting strategies to guide LLMs in generating code that adheres to the Singleton design pattern. They evaluated 13 LLMs using four prompting strategies (instructions, binary feedback, extensive feedback, and few-shot learning) on 164 Java coding challenges from HumanEval-X. The study found that iterative binary feedback generally provides the best balance between Singleton alignment and code functionality, with specific models like Llama 3.3 and Qwen 3 (8B) showing significant improvements in both aspects using different prompting techniques.
Simple iterative feedback can dramatically improve LLMs' adherence to software design patterns like Singleton, suggesting a path to more maintainable and robust AI-generated code.
Large Language Models (LLMs) can generate functional source code from natural-language prompts, but often fail to consistently follow higher-level architectural structures or design patterns. Since LLMs are increasingly used in software engineering, their ability to apply established design principles to generated code is crucial to the long-term success of software products. Therefore, the goal of this paper is to identify strategies for guiding LLMs to incorporate design patterns into the generated source code. We designed a computational experiment to evaluate the ability of 13 LLMs to generate code that follows the Singleton design pattern, using four prompting strategies: instructions, binary automated feedback, extensive automated feedback, and extensive feedback with few-shot prompts, in 164 Java coding challenges from HumanEval-X. Our results shows that the optimal strategy to guide LLMs to include design patterns depends heavily on the type of model. Still, overall, iterative binary feedback provides the best alignment with Singleton while preserving or improving the code's functionality. With guiding with instructions, Llama 3.3 generated Singleton classes in 100% of cases and improved code functionality, increasing the number of tests passed by 34.1 percentage points. It achieved a similar result with guidance through instructions and binary feedback. Qwen 3 (8B) increased the alignment with Singleton to 99.2% and the functionality to 58.6% using binary feedback. Our result suggests that even simple strategies can be used to guide LLMs to use design patterns.