Search papers, labs, and topics across Lattice.
This paper presents seven actionable recommendations for efficiently and responsibly adopting LLMs in industrial software development, derived from a multi-case study of three organizations. The recommendations cover user preferences for AI assistants, stakeholder satisfaction in output evaluation, scoping LLM applicability, workflow impacts, human oversight mechanisms, and required practitioner skills. A complementary survey validates the perceived relevance of these recommendations among software practitioners.
Forget hype, focus on human oversight: this study reveals practical, actionable recommendations for actually integrating LLMs into software development workflows responsibly.
Context: Large language models (LLMs) are observed to have a significant positive impact on various software engineering (SE) activities. With improved accessibility, the adoption of powerful LLMs in industry has surged recently. However, there is a lack of actionable best practices for the efficient and responsible adoption of LLMs within industrial software settings. Objectives: We developed seven actionable recommendations to address this research gap. Methods: We conducted a multi-case study with three organisations that use LLMs within their SE activities and synthesised seven recommendations through qualitative thematic analysis. We conducted a complementary online survey with software practitioners from various industries to evaluate the perceived relevance of our recommendations. Results: Our results and recommendations focus on (i) users' preference to use LLMs as AI assistants, (ii) the importance of relevant stakeholders' satisfaction in the LLM-output evaluation, (iii) scoping the applicability of LLMs within SE tasks, (iv) the effect of LLMs on SE workflows, (v) the necessity and directions for developing human oversight mechanisms, and (vi) the necessary skills for practitioners for leveraging LLMs within SE. The online survey indicates a high level of agreement from the participants regarding the perceived relevance of the recommendations. Conclusion: We outline future research directions, including mapping the seven recommendations to the principles of the EU AI Act (AIA) in order to examine how they relate to the current regulatory compliance frameworks.
