Search papers, labs, and topics across Lattice.
This study introduces an LLM-based analysis pipeline for CI/CD workflows that enhances cyber-systems engineering by integrating repository enrichment, anti-pattern detection, stage mining, and recommendation generation. By analyzing 75,201 workflows from 34,225 GitHub projects, the pipeline uncovers 434,769 anti-pattern findings primarily related to reliability and maintainability, and reveals significant differences in stage usage across programming languages. The findings advocate for a more comprehensive approach to CI/CD observability that emphasizes diagnosis and human review rather than solely classifying workflow stages.
CI/CD workflows are riddled with reliability issues, with over 434,000 anti-patterns identified across thousands of projects, highlighting the urgent need for improved observability and recommendations.
CI/CD workflows have become executable operational policy: they decide what gets built, tested, released, and deployed, and they mediate how maintainers interact with delivery infrastructure. That makes them an important measurement point for cyber-systems engineering. Recent large language model (LLM) work shows that workflow stages can be recognized directly from configuration files, but stage labels alone do not tell us whether a workflow is brittle, unusual for its ecosystem, or worth revising first. We present an LLM-based CI/CD analysis pipeline that combines repository enrichment, anti-pattern detection, stage mining, and recommendation generation over a large GitHub corpus. Starting from 59,550 repositories with at least 1,000 stars, we identify 34,225 projects with CI/CD and collect 127,559 configuration files. Across 75,201 analyzed workflows, the anti-pattern detector reports 434,769 findings, dominated by reliability and maintainability issues. Across 59,906 configurations, stage usage differs significantly by language ($\chi^2 = 4168.88$, $p<0.001$, Cramer's $V = 0.063$), and domain analysis shows distinct operational profiles, including higher release and cache usage in mobile projects. For repository-level recommendation generation, few-shot prompting performs best overall, averaging 8.25 recommendations per repository with 96.1% YAML-valid snippets. Taken together, the results argue for CI/CD observability that combines diagnosis, context, and human review rather than treating workflow mining as a stage-classification problem alone.
