Do Machines Struggle Where Humans Do? LLM and Human Comprehension of Obfuscated Code

2026-06-30

AI Summary

This study investigates whether large language models (LLMs) exhibit human-like patterns of difficulty when comprehending obfuscated code. Drawing on insights from human code comprehension research, we systematically evaluate multiple LLMs under five levels of code obfuscation—including control-flow flattening and adversarial identifier renaming—and pinpoint failure points across atomic, block, relational, and macroscopic levels. We introduce the Block Model evaluation framework and employ Chain-of-Thought (CoT) reasoning traces to analyze model behavior. Our analysis reveals, for the first time, that models fine-tuned with reasoning capabilities demonstrate sensitivity to code complexity closely aligned with human performance, exhibiting a strong correlation between task difficulty and CoT length; in contrast, models fine-tuned solely with instructions or code show virtually no such correlation.

Abstract

While code obfuscation impairs human code comprehension, it remains unclear if large language models share these failure modes. Building directly on a recent human study of program comprehension under code obfuscation, we evaluate whether large language models share the failure modes that obfuscation induces in human programmers. Evaluating several LLMs with five obfuscation tiers using the Block Model, we localize comprehension failures at the atom, block, relational, and macro levels. We find that reasoning-tuned models demonstrate significant alignment with human difficulty patterns across experience levels, whereas instruction and coder-tuned models show near-zero correlation. Chain-of-Thought trace length tracks task difficulty across tasks. Results indicate that performance under control-flow flattening degrades in proportion to state-space complexity, while adversarial identifier renaming disrupts comprehension through the interaction of semantic displacement and identifier-level interference. These findings suggest that reasoning-tuned LLMs approximate human sensitivity to code complexity more effectively than instruction-tuned variants.

Problem

• code obfuscation
• large language models
• human comprehension
• program understanding
• failure modes

Innovation

• code obfuscation
• large language models
• human comprehension
• reasoning-tuned models
• Block Model