Search papers, labs, and topics across Lattice.
This paper introduces a mechanistic framework that utilizes paired internal computation graphs to analyze vulnerabilities in large language models (LLMs) under adversarial prompts and jailbreak attacks. By aligning computation graphs for both clean and attacked prompts, the authors uncover systematic changes in internal reasoning, such as the suppression of safety-relevant components and the emergence of attack-specific features. Their findings reveal that structural deviations in these graphs correlate strongly with unsafe behaviors, and targeted interventions can enhance model robustness, shifting the focus from descriptive attribution to causal diagnosis of failures.
Adversarial attacks can fundamentally alter LLM internal reasoning, revealing hidden vulnerabilities that can be directly addressed through causal interventions.
Large language models (LLMs) exhibit remarkable capabilities but remain highly vulnerable to adversarial prompts and jailbreak attacks. Existing approaches primarily analyze these failures through input-output behaviors or attribution methods, offering limited insight into how adversarial perturbations alter the model's internal reasoning. Consequently, the mechanisms underlying unsafe or incorrect behaviors remain poorly understood. We introduce a mechanistic framework for diagnosing LLM vulnerabilities using paired internal computation graphs, which represent prompt-specific inference as structured causal interactions among latent features. By constructing and aligning computation graphs for clean and attacked prompts, we reveal that adversarial attacks induce systematic transformations of internal reasoning, including suppression of safety-relevant components, emergence of attack-specific features, and rerouting of computation paths. Building on this representation, we propose a unified framework that (i) decomposes computation into invariant, suppressed, and emergent structures, (ii) identifies recurring vulnerability motifs associated with failure modes, and (iii) performs causal interventions on nodes, paths, and subgraphs to directly evaluate their contributions to attack success. This enables a transition from descriptive attribution to causal diagnosis of model failures. Experiments across multiple open-source LLMs and diverse adversarial and jailbreak benchmarks demonstrate that structural deviations in internal computation graphs strongly correlate with unsafe behaviors. Furthermore, targeted interventions on identified vulnerability motifs improve model robustness, establishing internal computation graphs as a principled foundation for understanding, diagnosing, and mitigating LLM vulnerabilities.
