Search papers, labs, and topics across Lattice.
This study investigates the prevalence and characteristics of code and comments generated by large language models (LLMs) in both company- and community-maintained repositories from 2021 to 2025. The findings reveal a decline in LLM-generated code over time, particularly in test cases, while comments remain stable but often lack grammatical correctness. Notably, company-maintained repositories exhibit a higher incidence of LLM-generated content, yet only a minor fraction of human-identified bugs are linked to this generated code, suggesting a complex relationship between LLM usage and software quality.
LLM-generated code is declining in prevalence, yet company repositories still show a surprising reliance on it despite minimal bug association.
The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization. Reports from large technology companies showed that around 20% to 30% of their code are generated by LLMs. However, there remains skepticism about the practical usage of LLM-generated code and comments, such as concerns on more time for debugging the generated code and the unnaturalness of the generated comments. In this paper, we study the code and comments detected as likely to be generated by LLMs and their characteristics, the differences between company- and community-maintained repositories, and how likely bugs are associated with LLM-generated code. We conduct extensive experiments on active company- and community-maintained repositories from 2021 to 2025 using various tools and techniques that detect code and comments generated by LLMs. Based on our detector-based proxy analysis, the results suggest that code detected as likely to be generated by LLMs decreased over time and appeared frequently in test cases, while that of comments remains relatively stable. Proxy results further suggest that code detected as likely to be generated by LLMs shows substantial intra-repository code clones, whereas comments exhibit a relatively low proportion of grammatically correct sentences. In addition, the company-maintained repositories show a higher percentage of code and comments detected as likely to be generated by LLMs, and only a small percentage of the human-labelled bugs are detected as being likely associated with LLM-generated code.
