Search papers, labs, and topics across Lattice.
This paper introduces a lightweight probing method to efficiently estimate the downstream performance of LLMs during training. The probes are trained to predict a checkpoint's performance on downstream tasks (pass@1) using the LLM's internal representations as input. Experiments on OLMo3-7B show the probes achieve high accuracy (AUROC > 0.75) and significantly reduce evaluation latency compared to traditional generative evaluation.
Skip the costly generative evals: a simple probe trained on internal LLM representations can accurately predict downstream task performance during training, slashing evaluation time from an hour to just three minutes.
The paradigm of scaling Large Language Models (LLMs) in both parameter size and test time has pushed the boundaries of AI capabilities, but at the cost of making the traditional generative evaluation paradigm prohibitively expensive, therefore making the latency of LLM's in-training downstream performance evaluation unbearable. However, simple metrics like training loss (perplexity) are not always correlated with downstream performance, as sometimes their trends diverge from the actual task outcomes. This dilemma calls for a method that is computationally efficient and sufficiently accurate in measuring model capabilities. To address this challenge, we introduce a new in-training evaluation paradigm that uses a lightweight probe for monitoring downstream performance. The probes take the internal representations of LLM checkpoints (during training) as input and directly predict the checkpoint's performance on downstream tasks measured by success probability (i.e., pass@1). We design several probe architectures, validating their effectiveness using the OLMo3-7B's checkpoints across a diverse set of downstream tasks. The probes can accurately predict a checkpoint's performance (with avg. AUROC$>$0.75), have decent generalizability across checkpoints (earlier predicts later), and reduce the computation latency from $\sim$1 hr (using conventional generative evaluation method) to $\sim$3 min. In sum, this work presents a practical and scalable in-training downstream evaluation paradigm, enabling a more agile, informed, and efficient LLM development process.
