On Predicting the Post-training Potential of Pre-trained LLMs

2026-05-12

AI Summary

Existing benchmarks struggle to accurately assess the post-training potential of large language models in complex open-ended scenarios, leading to inefficient model selection. This work proposes RuDE, a novel framework that introduces a 4C taxonomy and a contrastive discrimination mechanism to construct fine-grained violation-aligned contrastive samples based on explicit scoring criteria. By leveraging rule-driven contrast pair generation, response discrimination modeling, and reinforcement learning validation—without relying on the model’s generative capabilities—RuDE achieves high-precision prediction of post-training performance. The method demonstrates over 90% correlation with actual post-training outcomes, successfully identifying multiple high-performing smaller models and substantially improving model selection efficiency while conserving computational resources.

Abstract

The performance of Large Language Models (LLMs) on downstream tasks is fundamentally constrained by the capabilities acquired during pre-training. However, traditional benchmarks like MMLU often fail to reflect a base model's plasticity in complex open-ended scenarios, leading to inefficient model selection. We address this by introducing a new task of predicting post-training potential - forecasting a base model's performance before post-training. We propose RuDE (Rubric-based Discriminative Evaluation), a unified framework that bypasses the generation gap of base models by leveraging response discrimination. Guided by our systematic 4C Taxonomy, RuDE constructs controlled contrastive pairs across diverse domains by fine-grained rubric violations. Extensive experiments demonstrate a correlation greater than 90% with post-training performance. Crucially, validation via Reinforcement Learning (RL) confirms that RuDE effectively identifies high-potential smaller models that outperform larger counterparts, offering a compute-efficient mechanism for foundation model development.

Problem

• post-training potential
• pre-trained LLMs
• model selection
• plasticity
• downstream performance

Innovation

• post-training potential prediction
• response discrimination
• RuDE
• 4C Taxonomy
• foundation model selection