Search papers, labs, and topics across Lattice.
This paper conducts a feature-level mechanistic analysis of backdoor attacks on large language models (LLMs) using sparse autoencoders (SAEs) to understand the fragmentation of existing defenses against dirty-label and clean-label attacks. By categorizing backdoor-induced logit shifts into four feature roles鈥攊nteraction, suppressed, mixed, and weight-modified鈥攖he authors reveal systematic encoding differences that explain the ineffectiveness of current defenses. The proposed inference-time feature clamping significantly reduces attack success rates while maintaining benign-task performance, demonstrating the potential of SAE-based analysis in guiding backdoor mitigation strategies.
Fragmented defenses against LLM backdoor attacks stem from fundamental differences in how clean and dirty-label attacks manipulate model features.
Backdoor attacks pose a serious threat to large language models (LLMs), but existing defenses remain fragmented, failing to pro?vide unified defense against both dirty-label and clean-label attacks. To investigate why such fragmentation arises, we present the first systematic feature-level mechanistic analysis of LLM backdoors using sparse autoencoders (SAEs). Starting from a 2 x 2 comparison of clean and poisoned models on clean and triggered inputs, we trace backdoor-induced logit shifts to high-contributing SAE features and categorize them into four roles: interac?tion, suppressed, mixed, and weight-modified features. This taxonomy reveals system?atic encoding differences: dirty-label back?doors are dominated by isolated interaction features, whereas clean-label backdoors rely more on heterogeneous mixtures of mixed and weight-modified features. These differ?ences explain why existing defenses remain fragmented across attack paradigms. We val?idate this hypothesis through inference-time feature clamping, which reduces ASR to at most 10.8% in most dirty-label settings and at most 15.4% in the majority of clean-label settings, while preserving benign-task perfor?mance. These results show that SAE-based analysis can explain defense fragmentation and guide interpretable backdoor mitigation.
