Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the propagation of soft errors in large language model (LLM) inference using a novel fault-injection framework called LLMFI. By systematically injecting faults into three open-weighted LLMs across thirteen diverse tasks, the authors uncover critical patterns of vulnerability and provide 17 key insights into how errors impact model performance. The findings also propose four low-overhead strategies for enhancing reliability in LLMs through software modifications, which are crucial for their integration into high-performance computing workflows.
Error propagation in LLMs reveals critical vulnerabilities that could undermine their reliability in high-stakes applications.
Large language models (LLMs) are increasingly integrated into high-performance computing (HPC) workflows, accelerating scientific discovery through diverse perspectives such as code generation and domain-specific decision-making. Yet, how soft errors propagate and affect LLM inference remains largely unexplored. To bridge this gap, we present a comprehensive study on error propagation in LLM inference, enabled by our proposed LLMFI, a configurable and deterministic fault-injection framework. Using LLMFI, we systematically inject faults across three open-weighted LLMs and thirteen representative tasks, covering reasoning, multilingual, mathematical, and coding domains. In addition, we conduct fine-grained case studies that reveal critical vulnerability patterns. Overall, our study yields 17 takeaways that advance the understanding of error propagation in LLM inference and introduces four low-overhead directions to improve reliability through software-only modification, offering practical guidance for future error detection and mitigation.