Dynamic N:M Fine-Grained Structured Sparse Attention Mechanism-Reference-Cited by-同舟云学术

Dynamic N:M Fine-Grained Structured Sparse Attention Mechanism

Published:2023-02-21 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming
language:
Short-container-title:

Author:

Chen Zhaodong¹,Qu Zheng¹,Quan Yuying¹,Liu Liu²,Ding Yufei¹,Xie Yuan³

Affiliation:

1. UC Santa Barbara, Santa Barbara, CA, USA

2. Rensselaer Polytechnic Institute, Santa Barbara, CA, USA

3. ALIBABA DAMO Academy, Santa Barbara, CA, USA

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3572848.3577500

Reference32 articles.

1. Iz Beltagy , Matthew E Peters , and Arman Cohan . 2020 . Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 (2020). Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 (2020).

2. Tianlong Chen , Yongduo Sui , Xuxi Chen , Aston Zhang , and Zhangyang Wang . 2021 . A unified lottery ticket hypothesis for graph neural networks . In International Conference on Machine Learning. PMLR, 1695--1706 . Tianlong Chen, Yongduo Sui, Xuxi Chen, Aston Zhang, and Zhangyang Wang. 2021. A unified lottery ticket hypothesis for graph neural networks. In International Conference on Machine Learning. PMLR, 1695--1706.

3. Zhaodong Chen Zheng Qu Liu Liu Yufei Ding and Yuan Xie. 2021. Efficient Tensor Core-Based GPU Kernels for Structured Sparsity under Reduced Precision. (2021). Zhaodong Chen Zheng Qu Liu Liu Yufei Ding and Yuan Xie. 2021. Efficient Tensor Core-Based GPU Kernels for Structured Sparsity under Reduced Precision. (2021).

4. Rewon Child , Scott Gray , Alec Radford , and Ilya Sutskever . 2019. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509 ( 2019 ). Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. 2019. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509 (2019).

5. Krzysztof Marcin Choromanski , Valerii Likhosherstov , David Dohan , Xingyou Song , Andreea Gane , Tamas Sarlos , Peter Hawkins , Jared Quincy Davis , Afroz Mohiuddin , Lukasz Kaiser , David Benjamin Belanger , Lucy J Colwell , and Adrian Weller . 2021 . Rethinking Attention with Performers. In International Conference on Learning Representations. https://openreview.net/forum?id=Ua6zuk0WRH Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, David Benjamin Belanger, Lucy J Colwell, and Adrian Weller. 2021. Rethinking Attention with Performers. In International Conference on Learning Representations. https://openreview.net/forum?id=Ua6zuk0WRH

Cited by 5 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Jigsaw: Accelerating SpMM with Vector Sparsity on Sparse Tensor Core;Proceedings of the 53rd International Conference on Parallel Processing;2024-08-12

2. A Survey of Text-Matching Techniques;Information;2024-06-05

3. FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs;Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays;2024-04

4. Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Design;IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems;2024-02

5. VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores;Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis;2023-11-11