Search papers, labs, and topics across Lattice.
This paper introduces a hybrid compression method that integrates pruning and quantization with Mixture of Experts (MoE) to optimize neural networks for resource-constrained environments. By first applying compression techniques to reduce model size and then utilizing MoEs to route the compressed models, the approach maintains performance while enhancing inference efficiency. Experimental results demonstrate significant reductions in FLOPs and parameters with only a minimal drop in accuracy across various benchmark datasets.
Achieving substantial model compression with negligible accuracy loss could redefine deployment strategies for neural networks on edge devices.
Deep neural networks have witnessed remarkable advancements in recent years and have become integral to various applications. However, alongside these developments, training and deployment of neural network models on embedding and edge devices face significant challenges due to limited memory and computational resources. These problems can be addressed with deep neural network compression, which involves a trade-off between model size and performance. In this paper, we propose a novel method for model compression through two phases. First, we utilize model compression techniques, such as pruning and quantization, to significantly reduce the model size. Then, we use Mixture of Experts to route the previously compressed models to enhance performance while maintaining a balance in inference efficiency. MoEs consist of multiple expert models (i.e., compressed models) that are moderately sized and deliver stable performance. Experimental results on several benchmark datasets show that our method successfully compresses CNN models which achieves substantial reductions in FLOPs and parameters with a negligible accuracy drop.
