Fast and Accurate Probing of In-Training LLMs' Downstream Performances

2026-04-01

AI Summary

This work addresses the challenge in large language model (LLM) training where traditional generative evaluation is computationally expensive and training loss poorly correlates with downstream performance. To overcome this, the authors propose a lightweight probing mechanism that predicts a model’s pass@1 success rate on downstream tasks using its internal representations. This approach enables efficient and accurate estimation of downstream performance during training and generalizes across checkpoints. Experimental results demonstrate that the probe achieves an average AUROC above 0.75 and reduces evaluation time from approximately one hour to just three minutes per assessment. Validated on the OLMo-3B model across diverse downstream tasks, the method significantly enhances model development efficiency.

Abstract

The paradigm of scaling Large Language Models (LLMs) in both parameter size and test time has pushed the boundaries of AI capabilities, but at the cost of making the traditional generative evaluation paradigm prohibitively expensive, therefore making the latency of LLM's in-training downstream performance evaluation unbearable. However, simple metrics like training loss (perplexity) are not always correlated with downstream performance, as sometimes their trends diverge from the actual task outcomes. This dilemma calls for a method that is computationally efficient and sufficiently accurate in measuring model capabilities. To address this challenge, we introduce a new in-training evaluation paradigm that uses a lightweight probe for monitoring downstream performance. The probes take the internal representations of LLM checkpoints (during training) as input and directly predict the checkpoint's performance on downstream tasks measured by success probability (i.e., pass@1). We design several probe architectures, validating their effectiveness using the OLMo3-7B's checkpoints across a diverse set of downstream tasks. The probes can accurately predict a checkpoint's performance (with avg. AUROC$>$0.75), have decent generalizability across checkpoints (earlier predicts later), and reduce the computation latency from $\sim$1 hr (using conventional generative evaluation method) to $\sim$3 min. In sum, this work presents a practical and scalable in-training downstream evaluation paradigm, enabling a more agile, informed, and efficient LLM development process.

Problem

• in-training evaluation
• downstream performance
• large language models
• evaluation latency
• performance probing

Innovation

• in-training evaluation
• lightweight probing
• downstream performance prediction
• LLM checkpoints
• computational efficiency