Search papers, labs, and topics across Lattice.
This study presents an automated unit test (UT) authoring workflow for the Open-Source Silicon Initialization Library (openSIL) firmware, leveraging a large language model (LLM) within a multi-agent pipeline to address the challenges of fragile unit tests in low-level C firmware. By integrating automated test scaffold generation, library-aware stub creation, and an iterative compile-dispatch repair loop informed by build logs and line coverage, the approach achieved a high success rate in generating compilable UTs. The results indicate that the workflow produced compilable tests for 73 out of 76 functions, with mean line coverage reaching 98.8% when utilizing line coverage guidance, demonstrating significant improvements in efficiency and coverage for constrained firmware environments.
Automated pipelines can generate high-coverage unit tests for low-level firmware, achieving up to 98.8% line coverage with minimal manual intervention.
Validating changes in low-level C firmware is expensive because unit tests (UTs) are fragile under strict build constraints, where missing headers, unresolved symbols, and dependency mismatches frequently prevent compilation and linking. This study introduces an automated UT authoring workflow for the Open-Source Silicon Initialization Library (openSIL) firmware codebase maintained by Advanced Micro Devices (AMD) that reduces manual effort through a large language model (LLM) guided multi-agent pipeline. The workflow combines automated generation of test scaffolds, library-aware creation or reuse of stubs, mocks, and fakes, and an iterative compile-dispatch repair loop driven by build logs and line-coverage feedback. We evaluate the approach using compilation success, repair iterations, dispatch success, and line coverage, with time, cost, and token usage as secondary measures. Across 76 functions under test, the workflow generated compilable UTs for 73 functions. In a configuration without line coverage guidance or retrieval augmentation, mean line coverage reached 73.9%. On a 48-function subset evaluated under both configurations, mean line coverage reached 98.8% with line-coverage guidance alone and reached 94.7% when combined with vector-database retrieval. Results show that automated generation-and-repair pipelines can substantially improve UT creation efficiency and coverage for constrained firmware environments while reducing manual debugging effort.
