Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a progressive magnitude-based pruning method that allows for effective neural network sparsification in a single training cycle, addressing the limitations of traditional iterative pruning methods. By gradually increasing sparsity during training and updating pruning masks based on active weight magnitudes, the approach demonstrates superior performance on CIFAR-10 and MNIST datasets across various architectures. Notably, it achieves 95.12% accuracy on ResNet-18 at 72.9% sparsity, surpassing the performance of established methods like the Lottery Ticket Hypothesis (LTH) and SNIP at similar sparsity levels.
Achieving over 95% accuracy at nearly 73% sparsity in a single training cycle challenges the conventional wisdom of iterative pruning methods.
Neural network pruning reduces model size by removing less important parameters while aiming to preserve predictive performance. Although the Lottery Ticket Hypothesis (LTH) shows that sparse subnetworks can match dense networks when trained from suitable initializations, its iterative pruning procedure requires multiple complete training cycles. This work evaluates progressive magnitude-based pruning as a single-cycle alternative. The method gradually increases sparsity during training using a linear schedule and updates pruning masks based on active weight magnitudes. We conduct systematic experiments on CIFAR-10 and MNIST across ResNet, VGG-style, and LeNet architectures, comparing the proposed method with representative iterative and initialization-based pruning baselines, including LTH, SNIP, and GraSP. On CIFAR-10, the method achieves 95.12\% accuracy on ResNet-18 at 72.9\% sparsity, compared with 90.5\% reported for LTH. At extreme sparsity, it achieves 93.13\% accuracy on a VGG-like architecture at 97\% sparsity, compared with approximately 92.0\% for SNIP, and 93.44\% accuracy on VGG-19 at 97.97\% sparsity, compared with 92.19\% for GraSP at 98\% sparsity. A sparsity-accuracy analysis on ResNet-18 further shows that accuracy remains within 0.1 percentage points of the dense baseline across 70--85\% sparsity. These results indicate that progressive magnitude-based pruning provides an effective single-cycle approach for neural network sparsification under the evaluated settings.