Search papers, labs, and topics across Lattice.
This study evaluates the structural validity of microservice decompositions generated by OpenAI's LLM, focusing on their adherence to source code dependencies in the PetClinic and Bookstore systems. Using an automated validation pipeline that incorporates static dependency analysis, the authors compare zero-shot and few-shot prompting strategies, revealing that both approaches yield similar levels of structural adherence. The key finding emphasizes the importance of controlling for class-to-service mapping coverage in evaluating LLM-generated architectures to avoid misleading interpretations of quality differences.
Structural evaluations of LLM-generated microservices reveal that apparent differences in prompting strategies may mask underlying methodological biases rather than true architectural quality.
Decomposing monolithic systems into microservices is a key activity in software modernization. Although Large Language Models (LLMs) can generate semantically plausible decompositions from textual requirements, it remains unclear whether these proposals preserve the structural dependencies implemented in the source code. This paper evaluates the structural adherence of microservice decompositions generated by OpenAI o3 for the PetClinic and Bookstore systems. We propose an automated validation pipeline based on static dependency analysis and compare zero-shot and few-shot prompting using dependency preservation (TPD) and dependency violation (TVD) metrics. A robustness analysis was conducted to control for differences in class-to-service mapping coverage. After normalization, both prompting strategies produced equivalent structural adherence, achieving TPD values of 68.0% (PetClinic) and 83.3% (Bookstore). The findings demonstrate that structural evaluations of LLM-generated decompositions should explicitly control for mapping coverage, as apparent differences between prompting strategies may otherwise reflect methodological bias rather than genuine architectural quality.
