Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces AKRASIA, a stealthy backdoor attack targeting reasoning-based Code LLMs, which successfully executes malicious code while evading both automated defenses and human detection. By probing the victim model to create a code-level backdoor trigger and utilizing in-context learning alongside model unfaithfulness, AKRASIA achieves an impressive average attack success rate of 99.34% across six reasoning LLMs. The results highlight the critical vulnerability of LLMs to reasoning backdoors, with AKRASIA maintaining high accuracy and evading detection in a majority of tested scenarios.
A staggering 99.34% success rate for backdoor attacks on reasoning-based LLMs reveals a critical vulnerability in current AI systems.
We present AKRASIA, a stealthy, inference-time backdoor attack against reasoning-based Code LLMs. AKRASIA aims to achieve a backdoor target (e.g., malicious code execution) in reasoning LLMs while evading automated defenses and human inspection. To achieve this, AKRASIA probes the victim LLM to construct a code-level backdoor trigger. It then employs in-context learning for backdoor learning, and model unfaithfulness to conceal the backdoor trigger, and generate plausible reasoning. We evaluate AKRASIA using four backdoor targets six (6) reasoning LLMs, three coding tasks/datasets and three defense methods. AKRASIA has up to 99.34% average attack success rate on SOTA LLMs and mantains up to 97.23% average accuracy. AKRASIA evades the SOTA defense, retaining up to 98.82% average ASR in most (14/18) defense settings. It evades human inspection, successfully hiding the backdoor trigger and reasoning steps in up to 80% of settings. Our findings motivate the need to defend LLMs against reasoning backdoors.