Search papers, labs, and topics across Lattice.
This paper addresses the high out-of-distribution (OOD) variance observed in Looped Transformers, which are designed for variable-length algorithmic tasks but struggle with stability. By introducing stochasticity in the number of loops during training, the authors significantly reduce OOD variance and enhance prediction stability across different inference-time loop counts. Their findings reveal that employing a learned stochastic schedule, particularly through RL-Halting, can improve the accuracy-stability trade-off in various algorithmic tasks, suggesting a paradigm shift in how stopping criteria are approached in model training.
Stochastic training can transform Looped Transformers from brittle to robust, dramatically reducing OOD variance and improving performance across diverse tasks.
Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length algorithmic tasks. Although they can exhibit strong length generalization beyond the length of training sequences, this behavior is brittle, yielding high out-of-distribution (OOD) variance, even across well-performing in-distribution solutions. We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops. Introducing stochasticity into the number of loops during training sharply reduces OOD variance and stabilizes predictions across inference-time loop counts. To improve upon heuristic randomization schemes, we further analyze RL-Halting as a learned stochastic schedule and find that it generally improves the accuracy-stability trade-off. Across binary addition, Dyck-1, Unique Set, and Copy, learned stochastic stopping often improves this trade-off but can also stabilize a suboptimal computation. Our work suggests that "when to stop" should be treated as a training-time design choice, not merely an inference-time computation-allocation rule.
