Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the inefficiency of Sharpness-Aware Minimisation (SAM) due to its fixed perturbation radius ($ρ$) by introducing Curvature-Aware Adaptive Perturbation for SAM (CAP-SAM). CAP-SAM adaptively modulates the perturbation radius based on local curvature, allowing for a more effective exploration of the loss landscape and convergence to robust minima. Experimental results demonstrate that CAP-SAM consistently outperforms standard SAM and its variants across multiple datasets and architectures.
Adaptive perturbation radius in SAM, guided by local curvature, unlocks superior generalization by escaping sharp minima more effectively without sacrificing convergence speed.
Poor generalisation in deep neural networks is frequently associated with convergence to sharp minima. Sharpness-Aware Minimisation (SAM) improves generalisation of deep neural networks by minimising a combination of the training loss and the local sharpness of the loss landscape, biasing convergence towards flat minima. However, SAM relies on a fixed perturbation radius $ρ$ that ignores local curvature, leading to inefficient behaviour: the perturbation may be too small to escape poor basins defined by sharp local minima, or too large to allow convergence to solutions with good loss-sharpness tradeoff. We reinterpret $ρ$ as the intensity of the perturbation signal and propose Curvature-Aware Adaptive Perturbation for SAM (CAP-SAM), a geometry-aware method that modulates the perturbation radius based on a sharpness ratio, enabling a more effective search for robust minima on the loss landscape. We prove that CAP-SAM preserves the same convergence rate as standard SAM in non-convex stochastic settings. Extensive experiments indicate that our approach consistently outperforms SAM and its most prominent variants across diverse network architectures on CIFAR-10, CIFAR-100, and ImageNet.