Search papers, labs, and topics across Lattice.
This study investigates the comprehension of obfuscated code by large language models (LLMs) in comparison to human programmers, building on a recent human-centric analysis. By evaluating several LLMs across five obfuscation tiers using the Block Model, the authors identify specific failure modes at various comprehension levels and reveal that reasoning-tuned models align closely with human difficulty patterns, unlike instruction and coder-tuned models. The results indicate that reasoning-tuned LLMs are more sensitive to the complexities introduced by code obfuscation, particularly in terms of state-space complexity and semantic displacement.
Reasoning-tuned LLMs mirror human struggles with obfuscated code, revealing critical insights into model design and comprehension limits.
While code obfuscation impairs human code comprehension, it remains unclear if large language models share these failure modes. Building directly on a recent human study of program comprehension under code obfuscation, we evaluate whether large language models share the failure modes that obfuscation induces in human programmers. Evaluating several LLMs with five obfuscation tiers using the Block Model, we localize comprehension failures at the atom, block, relational, and macro levels. We find that reasoning-tuned models demonstrate significant alignment with human difficulty patterns across experience levels, whereas instruction and coder-tuned models show near-zero correlation. Chain-of-Thought trace length tracks task difficulty across tasks. Results indicate that performance under control-flow flattening degrades in proportion to state-space complexity, while adversarial identifier renaming disrupts comprehension through the interaction of semantic displacement and identifier-level interference. These findings suggest that reasoning-tuned LLMs approximate human sensitivity to code complexity more effectively than instruction-tuned variants.
