Jeff Gore

Optimizers can dramatically alter step-by-step training trajectories, but a conserved geometric invariant ($2\alpha_r + \alpha_t = 1$) locks every algorithm from SGD to Adam and Muon into the exact same $D^{-1/3}$ optimal data scaling wall.

Fixed exponents in neural scaling laws reveal that optimizing coefficients could unlock significant performance gains in large language models.

Unconditional image diffusion models can now perform continuous super-resolution without task-specific architectures or retraining, simply by varying the starting timestep.