Search papers, labs, and topics across Lattice.
This paper introduces Log-Insight, an automated incident diagnosis system designed for large-scale microservice environments, addressing the challenges posed by the overwhelming volume of log data generated during incidents. By employing a neuro-symbolic approach that mimics the structured investigation process of skilled Site Reliability Engineers (SREs), Log-Insight significantly reduces raw log events while maintaining critical failure signals. The system demonstrates impressive performance, achieving a mean reciprocal rank of 0.790 and correctly identifying the root cause within the top three hypotheses in over 90% of evaluated incidents, all within a minute of latency.
Log-Insight transforms incident diagnosis from a manual, time-consuming process into an efficient automated workflow, achieving over 90% accuracy in identifying root causes within a minute.
Diagnosing production incidents in large-scale microservice systems is time-critical for Site Reliability Engineers (SREs). A single 30-minute incident window in our deployment can generate over two million log lines--approximately 1.2 billion characters, far exceeding standard LLM context windows--making direct LLM-based Root Cause Analysis (RCA) infeasible. Existing approaches leave gaps: template-based parsers lack semantic anomaly reasoning, deep-learning detectors emit black-box binary signals, and LLM pipelines suffer context overflow and domain hallucination on raw telemetry. We present Log-Insight, an automated incident-diagnosis system deployed in production at Huawei. The core design principle automates the SRE's manual triage workflow: symbolic stages replicate the structured investigation a skilled SRE would perform--sampling, schema understanding, pattern clustering, and statistical anomaly ranking. This hands the LLM a compact, pre-ranked evidence dossier to synthesise into a hypothesis report. Our six-stage pipeline reduces millions of raw events by 1,000-7,000x while preserving statistically significant failure signals. Evaluated on 11 historical production incidents (110 runs, SRE-validated ground truth), Log-Insight achieves MRR = 0.790, returning the correct root cause within the top-3 hypotheses in over 90% of runs in under a minute of latency. We report systematic failure modes, active mitigations, and open research directions. The Forensic Evidence section--listing exact log templates and skew statistics--was consistently identified by operators as a key adoption factor, shifting the system's perceived role from opaque oracle to investigative assistant.
