Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To quantify how architectural knowledge is captured in practice, the authors built an automated text-mining and LLM-classification pipeline to audit Architecture Decision Records (ADRs) across ~550 open-source repositories. While lightweight documentation templates like MADR are widely recommended to preserve institutional memory, empirical evidence on whether developers actually record structural rationale has remained scarce. The analysis reveals that while teams consistently log technology selections and existence decisions, they systematically omit evaluated alternatives, decision drivers, and non-functional quality attributes.
Open-source teams routinely use architecture records to log what technologies they chose, yet systematically fail to document the decision drivers, quality trade-offs, and rejected alternatives required to understand why.
Architectural decision records (ADRs) have become a popular lightweight mechanism for documenting architectural knowledge in software projects. However, there is limited empirical evidence on the kinds of architectural concerns captured in ADRs and how well their contents align with established architectural knowledge concepts and documentation practices. In this paper, we propose an automated text-mining and classification approach for analyzing ADRs at scale. We apply this approach to a dataset of ADRs extracted from ~550 open-source repositories, combining topic modeling, LLM-based classification, and template compliance checks. Our analysis examines decision taxonomies and quality attributes, and the degree to which ADRs adhere to the MADR template. Our findings show that ADRs frequently capture existence, technology, and process-related decisions, while alternatives, decisions drivers, and some quality concerns remain under-documented. We also observe recurring mismatches between ADR contents and template sections. These insights into current documentation practices provide architects with valuable information to reflect on how ADRs are and should be used to effectively deal with architectural knowledge. Furthermore, our automated approach is adaptable to other architectural tasks.