Maxim Bolshim

Monitoring per-layer curvature during training, previously intractable for large models, is now feasible with a new stochastic estimator that requires only a single backward pass.

Forget monolithic Hessians: this work unveils a new lens into neural network training by decomposing the Hessian into layer-wise interactions, revealing curvature structures previously hidden.