Search papers, labs, and topics across Lattice.
This paper critiques the traditional Software Development Lifecycle (SDLC) by highlighting its inadequacy for data-intensive and AI-enabled systems, where behavior is influenced by the interplay of code, data, and learned models. It introduces a novel five-layer lifecycle framework that integrates DataOps, MLOps, and LLMOps, emphasizing the need for a closed-loop control approach to maintenance amid configuration drift. The authors synthesize existing literature and propose a research agenda aimed at empirically quantifying the transformation of the SDLC to better accommodate the complexities of modern AI systems.
The integration of data engineering and software engineering practices could fundamentally redefine how we approach the software lifecycle in AI systems.
The traditional Software Development Lifecycle (SDLC) assumes that system behavior is determined primarily by source code, allowing correctness to be specified, implemented, and verified through code-centric practices. Data-intensive and AI-enabled systems challenge this assumption because their behavior emerges from the interaction of code, data, and learned models, while performance may degrade as real-world conditions drift from training data. This paper examines how integrating data engineering and software engineering practices, operationalized through DataOps, MLOps, and LLMOps, reshapes the SDLC for these systems. We make four contributions. First, we synthesize literature across software engineering, data management, machine learning systems, and human-centered computing into a phase-structured account of lifecycle transformation spanning requirements, architecture, development, testing, deployment, monitoring, governance, and organization. Second, we provide a lightweight formalization in which system behavior is defined over code, data, and model configurations; requirements become evaluation-led specifications with probabilistic acceptance regions; and promotion is controlled through statistically grounded validation gates. Third, we develop an adaptive five-layer lifecycle framework comprising artifact, contract, gate, control, and governance layers, positioning maintenance as a closed-loop control problem under configuration drift. Fourth, we propose a conceptual research model linking data engineering integration to measurable lifecycle outcomes and critically assess the evidence base. While the direction of transformation is increasingly established, its magnitude remains insufficiently quantified. We conclude with a research agenda for an empirically grounded, adaptive SDLC for data- and AI-centric systems.
