Search papers, labs, and topics across Lattice.
This paper investigates the robustness of sparse neural networks with adaptive topology, specifically focusing on a three-layer sparse architecture (99% sparsity) followed by a dense layer. The authors train these networks on MNIST and Fashion MNIST, updating the sparse layers' topology between epochs to maintain accuracy with fewer weights. They find that adaptive topology enhances robustness against random link removal, adversarial attacks, and link weight shuffling, in addition to improving efficiency.
Adaptive rewiring of sparse neural networks doesn't just improve efficiency, it also makes them surprisingly resilient to adversarial attacks and other perturbations.
We investigate the robustness of sparse artificial neural networks trained with adaptive topology. We focus on a simple yet effective architecture consisting of three sparse layers with 99% sparsity followed by a dense layer, applied to image classification tasks such as MNIST and Fashion MNIST. By updating the topology of the sparse layers between each epoch, we achieve competitive accuracy despite the significantly reduced number of weights. Our primary contribution is a detailed analysis of the robustness of these networks, exploring their performance under various perturbations including random link removal, adversarial attack, and link weight shuffling. Through extensive experiments, we demonstrate that adaptive topology not only enhances efficiency but also maintains robustness. This work highlights the potential of adaptive sparse networks as a promising direction for developing efficient and reliable deep learning models.
