Rescuing ReRAM-based Neural Computing Systems from Device Variation

Abstract

Resistive random-access memory (ReRAM)-based crossbar array (RCA) is a promising platform to accelerate vector-matrix multiplication in deep neural networks (DNNs). There are, however, some practical issues, especially device variation, that hinder the versatile development of ReRAM in neural computing systems. The device variations include device-to-device variation (DDV) and cycle-to-cycle variation (CCV) that deviate the devise resistance in the RCA from their target state. Such resistance deviation seriously degrades the inference accuracy of DNNs. To address this issue, we propose a software-hardware compensation solution that includes compensation training based on scale factors (CTSF) and variation-aware compensation training based on scale factors (VACTSF) to protect the ReRAM-based DNN accelerator against device variation. The scale factors in CTSF can be flexibly set for reducing accuracy loss due to device variation when the weights programmed into RCA are determined. For effectively handling CCV, the scale factors are introduced into the training process for obtaining variation-tolerant weights by leveraging the inherent self-healing ability of DNNs. Simulation results based on our method confirm that the accuracy losses due to device variation on LeNet-5, ResNet, and VGG16 with different datasets are less than 5% under a large device variation by CTSF. More robust weights for conquering CCV are also obtained by VACTSF. The simulation results present that our method is competitive in comparison to other variation-tolerant methods.