Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces "Cordyceps," a data poisoning attack that covertly controls LLMs by embedding an information hiding scheme within the model's knowledge base through semantic associations. This allows attackers to encode and decode arbitrary malicious instructions using attacker-chosen phrases. Experiments across five LLMs demonstrate that Cordyceps outperforms heuristic prompt injection attacks by 40% and maintains high success rates (up to 93-98%) even after backdoor and prompt injection defenses are applied.
LLMs can be subtly backdoored to execute arbitrary commands via semantically-triggered "covert control attacks" that evade common defenses.
Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize. In this paper, we propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily through semantic associations between shared knowledge such as facts or concepts and attacker-chosen phrases. The induced hiding scheme can encode and decode arbitrary malicious instructions, thus revealing a new and subtle poisoning-induced vulnerability: covert control attacks. We precisely characterize covert control attacks and evaluate them across $5$ LLMs, $3$ backdoor defenses, and $4$ prompt injection defenses. With a small poisoned fraction, covert control attacks outperform heuristic-based prompt injection attacks in average attack success rate by about $40\%$ relative to clean fine-tuned models. They also circumvent defenses based on detection and fine-tuning, maintaining up to $93\%$ attack success rate after backdoor defenses and up to $98\%$ after prompt injection defenses.