Search papers, labs, and topics across Lattice.
This study analyzes interview sample sizes and saturation practices in empirical software engineering research by reviewing papers published between 2016 and 2025. The findings reveal a significant variation in sample sizes, with studies often relying on fewer than 12 interviewees, particularly in specialized contexts, while the most common sample size falls between 13 to 24 participants. Additionally, the research highlights inconsistencies in how saturation and sample adequacy are justified, indicating a need for improved methodological transparency in the field.
Many software engineering studies rely on fewer than 12 interviews, raising questions about the rigor of qualitative research in the field.
Background. Interview based studies are widely used in empirical software engineering to investigate human, organizational, and socio technical phenomena, yet interview sample adequacy and saturation are reported inconsistently across the literature. Aims. This paper investigates how interview sample adequacy and saturation are operationalized in empirical software engineering research. Method. We analyzed papers published between 2016 and 2025 across major software engineering venues, focusing on interview sample sizes, saturation discussions, and sample adequacy justifications. Results. Preliminary findings indicate substantial variation in sample sizes, from highly specialized small sample studies to broader investigations involving large interview datasets. Studies involving fewer than 12 interviewees were common and frequently associated with specialized industrial contexts or constrained organizational access. However, the most recurrent range was 13 to 24 interviewees, suggesting that moderate sized samples represent the most common configuration in empirical software engineering research. Saturation and sample adequacy justification were heterogeneous, with many studies relying on implicit or contextual reasoning rather than explicit methodological discussion. Conclusions. Our findings provide initial empirical insights into methodological reporting practices in interview based software engineering research and contribute to ongoing discussions regarding qualitative rigor and transparency.
