Search papers, labs, and topics across Lattice.
TELLER is a novel framework designed for non-intrusive root-cause analysis of large language model (LLM) inference, addressing the complexities of diagnosing issues across multiple layers of execution. By leveraging NVTX/CUPTI traces and service logs, TELLER reconstructs per-request call-chain trees and aligns execution steps with log data, while introducing a dependency-aware causal-context slice for enhanced analysis. Experimental results demonstrate that TELLER achieves over 80% reduction in trace length with a moderate vocabulary, maintaining high diagnostic accuracy across both horizontal and vertical execution views, thus providing a robust tool for LLM inference troubleshooting.
TELLER achieves over 80% trace length reduction while maintaining high diagnostic accuracy, revolutionizing root-cause analysis for LLM inference.
Large language model (LLM) inference has evolved from an offline workload into a continuously operated software service, yet root-cause analysis remains difficult because a single request spans the inference engine, Python/C++ backend, host CUDA APIs, GPU kernels, and distributed communication. Existing profilers expose raw timelines, while log-based diagnosis often misses cross-layer execution semantics and request-level structure. We present TELLER, a non-intrusive Trace- and Log-aware LLM inference Root-cause analysis framework. TELLER first collects NVTX/CUPTI traces and service logs without modifying model binaries, then reconstructs per-request call-chain trees and aligns log lines with the corresponding execution steps. We introduce a dependency-aware causal-context slice that preserves parent-child structure, temporal order, and communication relations, and a Trace Pair Encoding (TPE) tokenizer that compresses such slices into compact structural token sequences with parent, depth, and duration attributes. On top of these representations, TELLER combines numeric candidate localization with a multimodal root-cause model that jointly predicts abnormal steps, localizes suspicious operators, and generates natural-language explanations. Experiments on multi-node GPU inference workloads show a clear compression-accuracy trade-off: a moderate TPE vocabulary reduces per-step trace length by more than 80% while achieving the best overall performance on both horizontal (cross-node communication) and vertical (within-node execution stack) views, whereas more aggressive compression substantially degrades diagnosis quality. Further analyses under low-fault priors, strengthened baselines, modality ablations, explanation-quality checks, and tracing overhead show that TELLER provides a practical triage and evidence-localization substrate for LLM inference RCA.
