Shaobo Zhang

Matching effective learning rates across different training setups leads to remarkably consistent loss trajectories, challenging conventional wisdom about learning rate variability.

Smooth activation functions unlock smoothness adaptivity in constant-depth neural networks, achieving minimax-optimal statistical rates without needing depth increases like ReLU networks.