Search papers, labs, and topics across Lattice.
This paper systematically investigates the impact of cognitive heuristics on Large Language Models (LLMs) used for code vulnerability detection, revealing that these models are influenced by biases similar to those affecting human judgment. Through a controlled framework, the authors assess eight LLMs across three programming languages, demonstrating that all models exhibit varying degrees of susceptibility to heuristics such as framing, anchoring, and the halo effect. Notably, the study finds that vulnerabilities requiring semantic reasoning are more prone to these biases, with a proof-of-concept attack capable of suppressing up to 97% of detected vulnerabilities, underscoring a critical flaw in LLM-based detection systems.
LLMs are not just tools; they are susceptible to cognitive biases that can drastically undermine their effectiveness in detecting code vulnerabilities.
Researchers and practitioners increasingly apply Large Language Models (LLMs) for automated vulnerability detection. Recent work has shown that LLMs are susceptible to the same cognitive heuristics that bias human judgment. Yet, no work has investigated whether these heuristics affect a model's assessment of code vulnerabilities. In this paper, we present the first systematic exploration of cognitive heuristics in LLM-driven code vulnerability detection. We introduce a controlled framework that holds the code fixed and only varies the surrounding context to trigger three cognitive heuristics: the halo effect through author attribution, the framing effect through task objectives and consequences, and the anchoring effect through prior analysis results. Within this framework, we evaluate eight LLMs across three programming languages and perform both quantitative and code-level analyses. Our findings demonstrate that all evaluated models are susceptible to these heuristics. Cross-model average susceptibility is highest for framing at 33.2%, followed by anchoring at 23.5% and halo at 18.4%. Code-level analysis reveals that vulnerabilities that require semantic reasoning for detection are more susceptible to cognitive heuristics than those identifiable through pattern matching. Furthermore, models often change their verdict from safe to vulnerable based on the cognitive condition, without accurately identifying the actual vulnerability. To highlight the practical impact, we demonstrate a proof-of-concept black-box cognitive attack that can suppress up to 97% of previously detected vulnerabilities. These findings indicate that cognitive susceptibility is a consistent and exploitable property of LLM-based vulnerability detection.
