Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
This paper introduces Predictable Scaling (PS), a framework for efficiently exploring hypothesis spaces in recommendation model development by predicting full-scale performance from small-scale proxies. PS leverages composable scaling functions and curve fitting with calibrated uncertainty to project metrics to target scales, enabling significant compute reductions. Experiments show PS correctly ranks actionable hypotheses across diverse types with 79-88% compute savings on both task-specific and foundation models.
Rank actionable recommendation model hypotheses with 79-88% compute savings by predicting full-scale performance from small-scale proxies.
Evaluating candidate changes (hypotheses) to recommendation models requires full-scale training runs consuming thousands of GPU-hours, severely limiting the hypothesis space engineers can feasibly explore. We introduce Predictable Scaling (PS), a systematic framework that exploits empirically verified regularity conditions to predict full-scale model performance from small-scale proxies. PS constructs scaled-down variants via composable scaling functions along distinct resource axes, and projects metrics to the target scale via curve fitting with calibrated uncertainty estimates. We compare three prediction strategies of increasing expressiveness: even the simplest (raw proxy) achieves prediction error within the empirical noise floor of repeated-run variance; the most expressive (Gaussian Process) provides an indicative 15% relative MAE improvement and calibrated uncertainty. Evaluation on two ranking models of increasing complexity---a task-specific model and a foundation model---demonstrates that PS correctly ranks actionable hypotheses across diverse hypothesis types at 79-88% compute reductions.