Search papers, labs, and topics across Lattice.
This paper introduces a finetuning-based hardware-aware training algorithm specifically designed for deploying deep neural networks (DNNs) on ReRAM crossbar arrays, addressing challenges posed by I-V non-linearity and retention errors. By applying a range-shrunk sinh transformation and incorporating retention errors into a regularization loss, the method allows for robust model performance with minimal training overhead. Experimental evaluations show that the proposed approach maintains comparable accuracy to baseline models across various tasks, with less than 2% accuracy degradation on ImageNet and only a 1-point drop in F-1 score on the SQuAD v2 dataset.
Achieving near-baseline accuracy for large-scale models on ReRAM architectures with minimal retraining could revolutionize energy-efficient AI deployment.
Traditional CPU, GPU, and NPU architectures are increasingly limited by the von Neumann bottleneck. While In-Memory Computing (IMC) using ReRAM crossbar arrays offers a high-density, energy-efficient alternative, its practical deployment is constrained through their non-idealities. Existing hardware-aware training frameworks often require training from scratch, which is computationally prohibitive for modern large-scale models. In this work, we propose a finetuning-based hardware-aware training algorithm that enables robust DNN deployment on ReRAM with minimal training overhead. Our approach mitigates I-V non-linearity by applying a range-shrunk sinh transformation and incorporates retention errors directly into a regularization loss during the finetuning process. We evaluate our framework across models and tasks such as image classification and question-answering (QA). Experimental results demonstrate that our method achieves similar accuracy on large-scale models like ResNet18 and DeiT-Tiny as the base model. In-case of ImageNet for MobileNetV3 families the technique has only less than 2% accuracy degradation. Further, applying the technique on the SQuAD v2 dataset results in only 1 point degradation of F-1 score.
