NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

2026-06-25

AI Summary

This work addresses the lack of transparency in data construction and training protocols for post-training large language models, which hinders reproducibility and lightweight optimization. Building upon Qwen3-8B-base, we present the first fully open, ablation-driven end-to-end post-training pipeline featuring dual branches for general instruction following and complex reasoning. Our approach integrates data distillation, multi-dimensional filtering, difficulty-based curriculum learning, task categorization, and diversity-aware sampling. We systematically investigate replacing reinforcement learning with optimal prediction distillation from single or multiple teachers (OPD/MOPD), substantially reducing reliance on task-specific verifiers. Experiments show that the general branch improves average benchmark scores from 55.01 to 61.85, while the reasoning branch rises from 73.88 to 75.17. Notably, MOPD with only 10K samples yields a 4.18-point overall gain, revealing inherent trade-offs among instruction following, mathematical reasoning, code generation, and general knowledge capabilities.

Abstract

Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization. This work presents NebulaExp, a fully transparent, ablation-driven post-training pipeline built on Qwen3-8B-base, covering two orthogonal model branches: general instruct model and complex reasoning-specialized model. We curate a raw corpus of 3.84M multi-source SFT samples and a 200K verifiable RL candidate pool, and design an end-to-end data processing stack including response distillation, multi-dimensional cross-verification filtering, fine-grained difficulty grading, task classification and diversity-aware sampling. For the Instruct branch, our three-stage optimized supervised fine-tuning approach NebulaExp-Ins-SFT improves the average benchmark score from the 55.01 baseline of Qwen3-8B-nothink to 60.99. GRPO reinforcement learning then further elevates the average score to 61.85. For the Reasoning branch, medium-difficulty GRPO RL improves average reasoning score from 73.88 to 75.17. To address RL's dependency on task verifiers, we systematically investigate single-teacher and multi-teacher OPD (MOPD): utilizing merely 4K instruction-following samples and outperforms RL baseline by 3.26 points on IFEval with +4.43 average overall gain; MOPD fuses four domain-specialist teachers with merely 10K samples, lifting average performance by 4.18 over the base model. This report provides a fully reproducible empirical post-training recipe for 8B-scale LLMs, and comprehensively dissects the capability trade-offs among instruction adherence, mathematical reasoning, code generation and general knowledge.

Problem

• post-training alignment
• reproducibility
• data construction
• training recipes
• large language models

Innovation

• post-training
• ablation study
• GRPO
• OPD
• data curation