Search papers, labs, and topics across Lattice.
This study investigates the phenomenon of "Blind Obedience" in code language models, where models follow incorrect instructions despite recognizing them as erroneous. Through four experiments utilizing the RunBugRun dataset, the authors demonstrate that this compliance leads to the introduction of irrecoverable semantic errors that persist even through iterative repair attempts. The results reveal that code language models can inadvertently exacerbate issues rather than resolve them, highlighting a critical flaw in their deployment in production environments.
Code LLMs can recognize incorrect instructions but still follow them, leading to irrecoverable semantic errors that defy traditional evaluation metrics.
Code language models are now trusted collaborators in production workflows for debugging, refactoring, and iterative repair, and every benchmark that evaluates them assumes the instructions they act on are correct. We study what happens when that assumption breaks. We evaluate code language models across four experiments designed to assess whether models resist or obey incorrect instructions in single-pass and iterative repair settings, using the RunBugRun dataset of algorithmic Python problems with deterministic test cases. Our findings reveal a striking behavioral pattern: models correctly identify an incorrect instruction as wrong, then follow it anyway. This compliance unknowingly introduces errors beyond the original bug, and the corrupted code state cannot be recovered through subsequent self-guided iterative repair, which fails to converge across passes. We term this Blind Obedience, characterize the Ghost (Unknown) Errors it introduces, quantify the proportion of cases where semantic corruption proves irrecoverable, and show that extended reasoning cannot reverse it. These findings surface behavioral properties invisible to pass-rate evaluation, with direct consequences for code language models deployed in production settings.
Microsoft Research