Search papers, labs, and topics across Lattice.
This paper presents an empirical study of how pre-trained models (PTMs) are used and updated as dependencies in downstream software systems, comparing their evolution to that of traditional software libraries. Analyzing 4,988 releases across 323 GitHub repositories, the study finds that PTMs are added later in project lifecycles, changed less frequently, and evolve proactively for capability expansion, unlike libraries. The key result is that PTMs exhibit a qualitatively distinct evolutionary pattern compared to traditional software libraries, driven by factors like capability expansion and testing uncertainty.
Pre-trained models in software projects aren't just libraries; they're sticky dependencies that accumulate over time and evolve based on capability, not just bug fixes.
Modern software systems have transitioned from purely code-based architectures to AI-integrated systems where pre-trained models (PTMs) serve as permanent dependencies. However, while the evolution of traditional software libraries is well-documented, we lack a clear understanding of how these"PTM dependencies"change over time. Unlike libraries, PTMs are characterized by opaque internals and less standardized, rapidly evolving release cycles. Furthermore, their multi-role nature enables developers to treat individual instances of a single PTM as separate functional dependencies based on their specific downstream tasks. This raises a critical question for software maintenance: do PTMs change like standard software libraries or do they follow a divergent pattern? To answer this, we present the first empirical study of downstream PTM changes, analyzing a comprehensive dataset of 4,988 releases across 323 GitHub OSS repositories that reuse open-source PTMs. Using traditional software libraries as a baseline, we find that PTMs follow a qualitatively distinct pattern. PTMs are typically added late in the project life-cycle and tend to accumulate rather than be replaced as a project matures. Our findings show that PTM changes are three times less frequent (406 of 2,814 release transitions) than library changes. PTM changes are also less routinely documented, but more likely to carry explicit rationale. Unlike libraries, which evolve reactively, PTM evolution is proactively driven by capability expansion, with a unique documented rationale of PTM testing uncertainty. Our work calls for a rethinking of how PTMs are tracked and managed as dependencies in modern software engineering.
