Search papers, labs, and topics across Lattice.
The paper reframes pretraining as creating a distribution over parameter vectors that already contains task-specific experts, rather than just a starting point for fine-tuning. It finds that in large, well-pretrained models, the density of these task-experts increases dramatically around the pretrained weights. The authors then demonstrate a simple, parallel post-training method involving random parameter perturbations and ensemble selection that achieves competitive performance compared to standard methods like PPO, GRPO, and ES.
Forget fine-tuning: task-specific experts are already hiding in the neighborhood of pretrained weights, and you can find them with random sampling.
Pretraining produces a learned parameter vector that is typically treated as a starting point for further iterative adaptation. In this work, we instead view the outcome of pretraining as a distribution over parameter vectors, whose support already contains task-specific experts. We show that in small models such expert solutions occupy a negligible fraction of the volume of this distribution, making their discovery reliant on structured optimization methods such as gradient descent. In contrast, in large, well-pretrained models the density of task-experts increases dramatically, so that diverse, task-improving specialists populate a substantial fraction of the neighborhood around the pretrained weights. Motivated by this perspective, we explore a simple, fully parallel post-training method that samples $N$ parameter perturbations at random, selects the top $K$, and ensembles predictions via majority vote. Despite its simplicity, this approach is competitive with standard post-training methods such as PPO, GRPO, and ES for contemporary large-scale models.
MIT CSAIL