An Energy-Efficient Transformer Processor Exploiting Dynamic Weak Relevances in Global Attention-Reference-Cited by-同舟云学术

An Energy-Efficient Transformer Processor Exploiting Dynamic Weak Relevances in Global Attention

Published:2023-01 Issue:1 Volume:58 Page:227-242
ISSN:0018-9200
Container-title:IEEE Journal of Solid-State Circuits
language:
Short-container-title:IEEE J. Solid-State Circuits

Author:

Wang Yang¹^ORCID,Qin Yubin¹^ORCID,Deng Dazheng¹,Wei Jingchuan¹^ORCID,Zhou Yang¹,Fan Yuanqi¹,Chen Tianbao²,Sun Hao¹,Liu Leibo¹^ORCID,Wei Shaojun¹^ORCID,Yin Shouyi¹^ORCID

Affiliation:

1. School of Integrated Circuits, the Beijing Innovation Center for Future Chip, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China

2. TsingMicro Technology, Beijing, China

Funder

NSFC

National Key Research and Development Program

Beijing National Research Center for Information Science and Technology

Beijing Advanced Innovation Center for Integrated Circuits

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Subject

Electrical and Electronic Engineering

Link

http://xplorestaging.ieee.org/ielx7/4/9999561/09927337.pdf?arnumber=9927337

Reference36 articles.

1. An Energy-Efficient Deep Convolutional Neural Network Training Accelerator for In Situ Personalization on Smart Devices

2. Exploration of Benes Network in Cryptographic Processors: A Random Infection Countermeasure for Block Ciphers Against Fault Attacks

3. An 8.93 TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity for On-Device Speech Recognition

4. SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training

5. 9.1 A 7nm 4-Core AI chip with 25.6TFLOPS hybrid FP8 training, 102.4TOPS INT4 inference and workload-aware throttling;agrawal;IEEE Int Solid-State Circuits Conf (ISSCC) Dig Tech Papers,2021

Cited by 11 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. MECLA: Memory-Compute-Efficient LLM Accelerator with Scaling Sub-matrix Partition;2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA);2024-06-29

2. A 3.55 mJ/frame Energy-efficient Mixed-Transformer based Semantic Segmentation Accelerator for Mobile Devices;2024 IEEE International Symposium on Circuits and Systems (ISCAS);2024-05-19

3. Inner Product Accelerating Scheme Based on RRAM Array for Attention-Mechanism Neural Network;IEEE Electron Device Letters;2024-03

4. Ayaka: A Versatile Transformer Accelerator With Low-Rank Estimation and Heterogeneous Dataflow;IEEE Journal of Solid-State Circuits;2024

5. Hardware-Efficient SoftMax Architecture With Bit-Wise Exponentiation and Reciprocal Calculation;IEEE Transactions on Circuits and Systems I: Regular Papers;2024