Search papers, labs, and topics across Lattice.
This paper introduces a scalable workflow for analyzing log data generated by high-performance computing (HPC) systems, addressing the challenges posed by the volume and complexity of such logs. By employing advanced pattern-matching techniques and utilizing a finite-state automaton for automated detection, the authors systematically identify frequent error patterns and correlate them with job logs to uncover insights into system performance and potential failures. The results indicate that this method significantly enhances real-time monitoring capabilities and contributes to the resilience of HPC environments.
Unlocking the hidden patterns in HPC log data can drastically improve system resilience and predictive maintenance strategies.
Modern supercomputers housed in High Performance Computing (HPC) environments generate massive volumes of log data daily, revealing intricate information and performance metrics about these complex systems. The sheer size and heterogeneous nature of HPC logs, especially text data, pose significant challenges for traditional analytical techniques. Consequently, more complex workflows are necessary for pattern extraction when analyzing these logs, enabling the discovery of underlying patterns and anomalies that may indicate system faults and help predict future failures and inefficiencies. Our log analysis workflow investigates a combination of advanced pattern-matching and mining techniques applied to HPC log analysis. By systematically identifying frequent log patterns and pattern sequences in log messages and storing them in a finite-state automaton, such as the Aho-Corasick automaton, our workflow enables automated detection of frequent errors and fault events. To extract these patterns and sequences, we leverage information about system hierarchy and message priority. We then correlate and cluster the identified error sequences with job logs, revealing groups of applications with similar or dissimilar error signatures. This approach yields insights that inform improvements and guide real-time monitoring efforts. Our research establishes that pattern mining is vital for unlocking the full potential of log data by enabling real-time analysis and contributing to more resilient, scalable HPC systems. We demonstrate the effectiveness of our approach through summary statistics and a case study on an exascale-class system supercomputer.
