Structural Anchors and Reasoning Fragility:Understanding CoT Robustness in LLM4Code

2026-04-13

AI Summary

This study investigates the intrinsic mechanisms underlying the reasoning robustness of Chain-of-Thought (CoT) prompting in large code generation models, uncovering the root causes of its performance instability under input perturbations. Through large-scale controlled experiments on six models across the MHPP and BigCodeBench benchmarks—combined with character-, word-, and sentence-level perturbations and token-level uncertainty tracking—the work identifies three structural anchors (“reasoning-to-code translation,” “symbolic commitment,” and “algorithmic articulation”) and three trajectory distortion patterns that collectively explain CoT failure modes. The findings reveal that CoT robustness is jointly influenced by model architecture, task structure, and prompt specificity, and that early-stage uncertainty serves as an effective predictor of trajectory instability, offering both theoretical insights and practical guidelines for designing more robust reasoning-based code generators.

Technology Category

• Reasoning under Uncertainty: Causality
• Natural Language Processing: Safety and Robustness
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Chain-of-Thought (CoT) prompting is widely used to elicit explicit reasoning from large language models for code (LLM4Code). However, its impact on robustness and the stability of reasoning trajectories under realistic input perturbations remains poorly understood. Prior work has largely evaluated CoT through final correctness, leaving a critical gap in understanding how CoT reshapes internal uncertainty dynamics and why it sometimes harms rather than helps code generation. We suggest that CoT is not uniformly beneficial; instead, its robustness depends on whether perturbations destabilize structurally sensitive commitment points along the reasoning-to-code trajectory. We conduct a controlled, large-scale empirical study of CoT across six models and two code benchmarks (MHPP and BigCodeBench), subjecting task docstrings to systematic character-, word-, and sentence-level perturbations. We instrument full generation traces with token-level uncertainty and define three novel structural anchors: reasoning-code transition, symbolic commitment, and algorithmic articulation. Findings: (1) CoT does not yield uniform performance or robustness gains: its benefits are contingent on model family, task structure, and prompt explicitness. (2) CoT and No-CoT exhibit distinct robustness profiles, with different perturbation families triggering different failure modes. (3) We identify three recurrent trajectory deformations--Lengthening, Branching, and Simplification--that systematically emerge when perturbations interact with structural anchors and explain failure patterns. (4) Early-stage uncertainty serves as a reliable diagnostic signal for localizing where trajectory instability begins around sensitive anchors. These results provide a unified explanation for CoT's mixed performance and suggest design principles for building more robust reasoning-based code generators.

Problem

• Chain-of-Thought
• robustness
• code generation
• reasoning trajectory
• input perturbations

Innovation

• structural anchors
• Chain-of-Thought robustness
• LLM4Code
• reasoning trajectory
• uncertainty dynamics