Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse

2026-07-05

AI Summary

This study investigates the “blind compliance” behavior of code large language models (LLMs), which execute erroneous instructions even when they recognize the errors, leading to irreversible “ghost errors” and semantic collapse. Through four sets of experiments on algorithmic Python problems from the RunBugRun dataset with deterministic test cases, the work systematically evaluates model responses in both single-attempt and multi-round repair scenarios. It formally defines and reveals the blind compliance phenomenon, quantifies the proportion of irrecoverable semantic failures, and demonstrates that extended reasoning fails to effectively mitigate such errors. The findings expose critical limitations of conventional pass-rate metrics and raise significant concerns about the reliability of code LLMs in production environments.

Abstract

Code language models are now trusted collaborators in production workflows for debugging, refactoring, and iterative repair, and every benchmark that evaluates them assumes the instructions they act on are correct. We study what happens when that assumption breaks. We evaluate code language models across four experiments designed to assess whether models resist or obey incorrect instructions in single-pass and iterative repair settings, using the RunBugRun dataset of algorithmic Python problems with deterministic test cases. Our findings reveal a striking behavioral pattern: models correctly identify an incorrect instruction as wrong, then follow it anyway. This compliance unknowingly introduces errors beyond the original bug, and the corrupted code state cannot be recovered through subsequent self-guided iterative repair, which fails to converge across passes. We term this Blind Obedience, characterize the Ghost (Unknown) Errors it introduces, quantify the proportion of cases where semantic corruption proves irrecoverable, and show that extended reasoning cannot reverse it. These findings surface behavioral properties invisible to pass-rate evaluation, with direct consequences for code language models deployed in production settings.

Problem

• Blind Obedience
• Code LLMs
• Semantic Collapse
• Incorrect Instructions
• Iterative Repair

Innovation

• Blind Obedience
• Semantic Collapse
• Ghost Errors
• Iterative Repair
• Code Language Models