DOTA: detect and omit weak attentions for scalable transformer acceleration-Reference-Cited by-同舟云学术

DOTA: detect and omit weak attentions for scalable transformer acceleration

Published:2022-02-22 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems
language:
Short-container-title:

Author:

Qu Zheng¹^ORCID,Liu Liu¹^ORCID,Tu Fengbin¹^ORCID,Chen Zhaodong¹^ORCID,Ding Yufei¹^ORCID,Xie Yuan¹^ORCID

Affiliation:

1. University of California at Santa Barbara, USA

Funder

NSF

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3503222.3507738

Reference55 articles.

1. Database-friendly random projections

2. SnaPEA: Predictive Early Activation for Reducing Computation in Deep Convolutional Neural Networks

3. Computer Architecture and Amdahl's Law

4. Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv:2005.14165. Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv:2005.14165.

5. End-to-End Object Detection with Transformers

Cited by 31 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Hardware–Software Co-Design Enabling Static and Dynamic Sparse Attention Mechanisms;IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems;2024-09

2. P²-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer;IEEE Transactions on Very Large Scale Integration (VLSI) Systems;2024-09

3. Mentor: A Memory-Efficient Sparse-dense Matrix Multiplication Accelerator Based on Column-Wise Product;ACM Transactions on Architecture and Code Optimization;2024-08-26

4. Raptor-T: A Fused and Memory-Efficient Sparse Transformer for Long and Variable-Length Sequences;IEEE Transactions on Computers;2024-07

5. MECLA: Memory-Compute-Efficient LLM Accelerator with Scaling Sub-matrix Partition;2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA);2024-06-29