Capacity-Dependent Effects of Data Selection for Reasoning

2026-08-13

AI Summary

This study addresses the oversight of model capacity and training duration in reasoning fine-tuning data selection. Through controlled mathematical reasoning experiments and learning dynamics analysis, we reveal that data selection efficacy is inherently capacity-dependent. We propose a Capacity-Constrained Distillation theoretical framework, uncovering a Fast-Fit/Slow-Gain pattern wherein small models benefit from high-likelihood data for rapid fitting, while large models gain more from low-likelihood data during extended training. By elucidating the synergistic mechanism between data difficulty and model capacity, this work provides both theoretical foundations and practical guidelines for adaptively formulating data selection strategies aligned with specific computational budgets.

Abstract

In reasoning supervised fine-tuning, candidate responses for the same instruction can differ substantially in how well they match the student's current distribution. Recent likelihood-based response selection methods suggest that responses closer to the student distribution provide more effective supervision, motivating the hypothesis that high-likelihood responses may generally be preferable for fine-tuning. In this paper, we revisit this intuition and show that the value of likelihood-based data selection depends critically on model capacity and training duration. Through controlled experiments on mathematical reasoning, using students ranging from 1.5B to 8B parameters and supervision generated by stronger teacher models, we observe a clear \emph{capacity-dependent} ``{\color{SMALLCOLOR}\textbf{Fast-Fit}} / {\color{LARGECOLOR}\textbf{Slow-Gain}}'' pattern. High-likelihood data provides faster and more stable early improvements, especially for smaller models, but low-likelihood data becomes increasingly beneficial for larger models when training is allowed to continue longer. To explain this phenomenon, we analyze learning dynamics, showing that small models often fail to absorb low-likelihood supervision and instead fall into shallow or repetitive behaviors, while larger models are better able to move toward the teacher distribution under such data. We further provide a capacity-constrained theoretical view of distillation that clarifies how data difficulty, data span, and student capacity jointly govern transfer. Overall, our findings show that effective data selection for reasoning should be aware of model capacity and computing budget rather than based on a single universal preference for high-likelihood supervision.

Problem

• Data Selection
• Reasoning
• Supervised Fine-Tuning
• Model Capacity
• Likelihood

Innovation

• Capacity-Dependent Data Selection
• Reasoning SFT
• Fast-Fit Slow-Gain
• Learning Dynamics
• Capacity-Constrained Distillation