From Brewing to Resolution: Tracing the Internal Lifecycle of Code Reasoning in LLMs

2026-06-16

AI Summary

Existing accuracy metrics fail to uncover the underlying mechanisms behind large language models’ differential performance on semantically equivalent code structures, such as loops, in code reasoning tasks. This work proposes a two-phase “Incubation–Resolution” lifecycle model for code reasoning and introduces a dual diagnostic framework that formally defines and empirically validates four resolution outcomes: Resolved, Overprocessed, Misresolved, and Unresolved. By integrating layer-wise linear probing with Context-Stripped Decoding (CSD), the study systematically evaluates sixteen mainstream models across six task categories, revealing that only 41.5% of samples are correctly resolved. Resolution success plummets to 2.5% when function call depth reaches three levels. Notably, the incubation phase consistently accounts for 24–42% of the reasoning process regardless of model capability, exposing diverse failure modes obscured by aggregate accuracy scores.

Abstract

Standard accuracy metrics cannot explain why LLMs handle variable tracking but fail on semantically equivalent loops. We study an internal lifecycle of code reasoning in which models first brew the answer, making it linearly recoverable many layers before it becomes self-decodable, and then diverge into one of four resolution outcomes: Resolved, Overprocessed, Misresolved, or Unresolved. Understanding this lifecycle matters because similar task accuracies can mask fundamentally different failure modes that surface-level evaluation cannot detect. We introduce a dual diagnostic framework pairing layer-wise linear probing with Context-Stripped Decoding (CSD) and apply it to six code-reasoning task families across 16 models spanning Qwen, Llama, and DeepSeek architectures. All four outcomes carry substantial mass in every task family: overall Resolved is only 41.5%, with multiple tasks below 30%. Controlled sweeps over structure, depth, and operators expose task-specific failure bottlenecks: Function Call Resolved plunges from 61.1% to 2.5% as call depth increases from one to three. Across architectures and scales, the brewing scaffold remains stable, with normalized brewing duration 24-42% across all 16 models, while resolution success varies with capability. This indicates that the scaffold is a stable empirical regularity across the tested decoder-only Transformer families, whereas resolution success covaries with capability, scale, and training. Code: https://github.com/euyis1019/llm-brewing

Problem

• code reasoning
• failure modes
• large language models
• variable tracking
• loop semantics

Innovation

• code reasoning
• internal lifecycle
• linear probing
• Context-Stripped Decoding
• failure modes