On the Step Length Confounding in LLM Reasoning Data Selection

2026-04-08

AI Summary

This study addresses a critical yet previously overlooked issue in naturalness-based data selection for large language model (LLM) reasoning: the “step-length confounding” problem, wherein existing methods that rely on average log-probability inadvertently favor samples with more reasoning steps rather than higher reasoning quality. The authors formally identify and name this phenomenon, demonstrating that low probability of the first token is a key source of scoring bias. To mitigate this, they propose two debiasing strategies—ASLEC-DROP, which excludes the first-token probability from the scoring, and ASLEC-CASL, a causally motivated debiased regression approach—both effectively disentangling reasoning quality from step length. Extensive experiments across four prominent LLMs and five reasoning benchmarks show that the proposed methods substantially alleviate step-length confounding and significantly improve data selection quality.

Abstract

Large reasoning models have recently demonstrated strong performance on complex tasks that require long chain-of-thought reasoning, through supervised fine-tuning on large-scale and high-quality datasets. To construct such datasets, existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalness-based selection methods to filter high-quality samples. Despite the proven effectiveness of naturalness-based data selection, which ranks data by the average log probability assigned by LLMs, our analysis shows that, when applied to LLM reasoning datasets, it systematically prefers samples with longer reasoning steps (i.e., more tokens per step) rather than higher-quality ones, a phenomenon we term step length confounding. Through quantitative analysis, we attribute this phenomenon to low-probability first tokens in reasoning steps; longer steps dilute their influence, thereby inflating the average log probabilities. To address this issue, we propose two variant methods: ASLEC-DROP, which drops first-token probabilities when computing average log probability, and ASLEC-CASL, which applies a causal debiasing regression to remove the first tokens' confounding effect. Experiments across four LLMs and five evaluation benchmarks demonstrate the effectiveness of our approach in mitigating the step length confounding problem.

Problem

• step length confounding
• reasoning data selection
• large language models
• naturalness-based selection
• average log probability

Innovation

• step length confounding
• reasoning data selection
• average log probability
• causal debiasing
• large language models