Search papers, labs, and topics across Lattice.
This paper introduces ADEPT, a hardware-aware framework designed to optimize the fine-tuning of Convolutional Neural Networks (CNNs) on Processing-in-Memory (PIM) architectures. By incorporating a novel metric that balances gradient-based sensitivity with the Energy-Delay Product (EDP) of the hardware, ADEPT enables adaptive training configurations that significantly reduce off-chip memory access and total trainable parameters. The results demonstrate that ADEPT achieves comparable predictive accuracy to traditional methods while enhancing energy efficiency during the fine-tuning process.
Fine-tuning CNNs on PIM architectures can be both energy-efficient and accurate, with ADEPT cutting off-chip memory access while maintaining performance.
Processing-in-memory-based (PIM) architectures have emerged as a promising solution for accelerating Convolutional Neural Network (CNN) workloads at the edge. Fine-tuning pre-trained CNNs is a common requirement to enhance the model predictive accuracy after deployment. However, the fine-tuning process is computational and memory-intensive, generating a significant amount of intermediate activations. This leads to frequent off-chip memory access, affecting the overall efficiency of the PIM accelerator. Existing fine-tuning strategies are agnostic to the underlying hardware, as they treat all layers equally. In this paper, we propose a hardware-aware framework called ADEPT to accelerate CNN fine-tuning on PIM architectures. Unlike prior fine-tuning methods, ADEPT adaptively trains the model considering both the training overhead and layer sensitivity. Specifically, ADEPT introduces a novel metric that quantifies the trade-off between a block's gradient-based sensitivity and its hardware architecture-specific Energy-Delay Product (EDP), producing platform-dependent fine-tuning configurations. Overall, ADEPT helps reduce the total trainable parameters and the off-chip data access during fine-tuning, while incurring minimal loss in predictive accuracy compared to full-parameter fine-tuning.
