Author:
Zhang Fan,Yang Li,Meng Jian,Seo Jae-sun,Cao Yu,Fan Deliang
Abstract
Recently, ReRAM crossbar-based deep neural network (DNN) accelerator has been widely investigated. However, most prior works focus on single-task inference due to the high energy consumption of weight reprogramming and ReRAM cells’ low endurance issue. Adapting the ReRAM crossbar-based DNN accelerator for multiple tasks has not been fully explored. In this study, we propose XMA2, a novel crossbar-aware learning method with a 2-tier masking technique to efficiently adapt a DNN backbone model deployed in the ReRAM crossbar for new task learning. During the XMA2-based multi-task adaption (MTA), the tier-1 ReRAM crossbar-based processing-element- (PE-) wise mask is first learned to identify the most critical PEs to be reprogrammed for essential new features of the new task. Subsequently, the tier-2 crossbar column-wise mask is applied within the rest of the weight-frozen PEs to learn a hardware-friendly and column-wise scaling factor for new task learning without modifying the weight values. With such crossbar-aware design innovations, we could implement the required masking operation in an existing crossbar-based convolution engine with minimal hardware/memory overhead to adapt to a new task. The extensive experimental results show that compared with other state-of-the-art multiple-task adaption methods, XMA2 achieves the highest accuracy on all popular multi-task learning datasets.
Reference52 articles.
1. Resistive random access memory (reram) based on metal oxides;Akinaga;Proc. IEEE,2010
2. Puma: A programmable ultra-efficient memristor-based accelerator for machine learning inference;Ankit,2019
3. A fully integrated reprogrammable memristor–cmos system for efficient multiply–accumulate operations;Cai;Nat. Electron.,2019
4. Emat: An efficient multi-task architecture for transfer learning using reram;Chen,2018
5. Neurosim: A circuit-level macro model for benchmarking neuro-inspired architectures in online learning;Chen;IEEE Trans. Comput. -Aided. Des. Integr. Circuits Syst.
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Heterogeneous Memory Integration and Optimization for Energy-Efficient Multi-Task NLP Edge Inference;Proceedings of the 29th ACM/IEEE International Symposium on Low Power Electronics and Design;2024-08-05
2. Efficient Multi-task Adaption for Crossbar-based In-Memory Computing;2022 56th Asilomar Conference on Signals, Systems, and Computers;2022-10-31