SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
This work investigates the universality and training dynamics of zero-shot reinforcement learning (Zero RL) across heterogeneous foundation models. Method: We systematically evaluate whether chain-of-thought (CoT) reasoning emerges directly from base models—without explicit CoT supervision—across ten open-source models spanning diverse architectures and scales. We introduce two key design principles: format reward shaping and query difficulty control, and integrate rule-based RL, implicit CoT supervision, joint monitoring of response length and verification behavior, and a cross-model training dynamics analysis framework. Contribution/Results: We observe, for the first time, a “reasoning insight moment” in non-Qwen small-scale models. Our analysis reveals a non-monotonic relationship between model scale and training dynamics. Experiments demonstrate significant improvements in reasoning accuracy and response length across most models. To foster reproducibility, we open-source all code, fine-tuned models, and analytical tools.