Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ModBench, a novel pipeline designed to mine Git repositories of Modelica libraries to create benchmark datasets of model snapshots. By filtering for human-authored, Modelica-relevant revisions and extracting simulation-eligible classes, ModBench produces a comprehensive dataset that includes 85,562 distinct class snapshots from the Modelica Standard Library's entire commit history. This work not only enhances empirical research capabilities in equation-based modeling but also provides traceability to original models and Git metadata, facilitating further studies in model evolution and automated processes.
ModBench reveals over 85,000 distinct Modelica class snapshots, transforming how researchers can analyze model evolution and development.
Research on equation-based cyber-physical systems modeling languages, such as Modelica, is constrained by the lack of curated benchmark datasets. This limits empirical insight into the evolution and development of models. We address this gap with ModBench, a pipeline that mines Git repositories of Modelica libraries to produce benchmark datasets of model snapshots. The pipeline (1) filters repository commits to retain human-authored, Modelica-relevant revisions; (2) extracts simulation-eligible classes; and (3) builds canonical representations of Modelica classes. For empirical validation, we applied ModBench to the Modelica Standard Library (MSL) and report the resulting dataset, spanning the full commit history (since Modelica language v3), with 85,562 distinct class snapshots, and links enabling traceability to original models and Git metadata. The dataset, its API, and the data generation pipeline are publicly available to support future research on model evolution analysis, compiler testing, and automated model repair or generation.