The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling

2026-06-28

AI Summary

This study investigates the performance degradation of large language models as reasoning depth increases and identifies task-dependent upper bounds on their reasoning capabilities. To this end, the authors introduce the Complexity Ceiling Benchmark (CCB), which modulates reasoning depth from 5 to 50 steps under fixed semantic conditions, evaluating model performance across three task categories: spatial state tracking, symbolic pointer manipulation, and transitive relational reasoning. Through controlled depth-scaling experiments, trajectory-level error analysis (TFBC), explicit state tracking, and McNemar’s tests, the study conducts 6,000 evaluations across five state-of-the-art models. It reveals distinct “complexity ceilings” across tasks: the strongest models maintain high accuracy (>92%) at 50 steps in the first two tasks, whereas relational reasoning collapses within 5 steps. Notably, 14.5% of correct answers stem from flawed reasoning trajectories, and the step of first error (k*) proves a stronger predictor of overall accuracy than model parameter count.

Abstract

We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays as the number of required sequential steps grows. CCB fixes the semantic content of a task and varies only its depth N in {5,...,50} across three structurally distinct regimes: grounded spatial state-tracking, abstract symbolic pointer manipulation, and transitive relational inference. Across 6,000 trials over five frontier and open-weight LLMs we find a consistent pattern of geometric per-step decay with widely separated domain ceilings: on the first two regimes the strongest models retain pd>0.92 across N=50; on the third every model collapses by N=5, with the best model's 50%-success horizon at H0.5~4.7 steps despite pd=0.863. A trace-level metric (TFBC) shows that 14.5% of correct answers across the benchmark are reached via incorrect intermediate reasoning. Forced verbose state-tracking does not move the ceiling (McNemar p=1.000), and the mean step at which reasoning first diverges, k*, predicts within-domain accuracy better than parameter count. CCB and the geometric decay model together reduce a model's long-horizon reasoning profile to one interpretable number per task family.

Problem

• sequential reasoning
• reasoning decay
• depth scaling
• language models
• reasoning benchmark

Innovation

• Complexity Ceiling Benchmark
• sequential reasoning
• depth scaling
• geometric decay
• reasoning fidelity