Affiliation:
1. Georgia Institute of Technology, Atlanta, GA
2. NVIDIA, Westford, MA
Abstract
A spatial accelerator’s efficiency depends heavily on both its mapper and cost models to generate optimized mappings for various operators of DNN models. However, existing cost models lack a formal boundary over their input programs (operators) for accurate and tractable cost analysis of the mappings, and this results in adaptability challenges to the cost models for new operators. We consider the recently introduced Maestro Data-Centric (MDC) notation and its analytical cost model to address this challenge because any mapping expressed in the notation is precisely analyzable using the MDC’s cost model.
In this article, we characterize the set of input operators and their mappings expressed in the MDC notation by introducing a
set of conformability rules
. The outcome of these rules is that any loop nest that is perfectly nested with affine tensor subscripts and without conditionals is conformable to the MDC notation. A majority of the primitive operators in deep learning are such loop nests. In addition, our rules enable us to automatically translate a mapping expressed in the loop nest form to MDC notation and use the MDC’s cost model to guide upstream mappers. Our conformability rules over the input operators result in a
structured mapping space
of the operators, which enables us to introduce a mapper based on our
decoupled off-chip/on-chip
approach to accelerate mapping space exploration. Our mapper decomposes the original higher-dimensional mapping space of operators into two lower-dimensional off-chip and on-chip subspaces and then optimizes the off-chip subspace followed by the on-chip subspace. We implemented our overall approach in a tool called
Marvel
, and a benefit of our approach is that it applies to any operator conformable with the MDC notation. We evaluated Marvel over major DNN operators and compared it with past optimizers.
Funder
NSF
U.S. Department of Energy’s National Nuclear Security Administration
Publisher
Association for Computing Machinery (ACM)
Subject
Hardware and Architecture,Information Systems,Software
Reference57 articles.
1. Edge TPU: Google’s Purpose-Built ASIC Designed to Run Inference at the Edge. Retrieved October 13, 2021 from;Google Cloud.;https://cloud.google.com/edge-tpu/
2. Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code
3. End to end learning for self-driving cars;Bojarski Mariusz;arXiv preprint arXiv:1604.07316,2016
Cited by
8 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. SecureLoop: Design Space Exploration of Secure DNN Accelerators;56th Annual IEEE/ACM International Symposium on Microarchitecture;2023-10-28
2. NeuroSpector: Systematic Optimization of Dataflow Scheduling in DNN Accelerators;IEEE Transactions on Parallel and Distributed Systems;2023-08
3. Graphene: An IR for Optimized Tensor Computations on GPUs;Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3;2023-03-25
4. DefT: Boosting Scalability of Deformable Convolution Operations on GPUs;Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3;2023-03-25
5. Sigma: Compiling Einstein Summations to Locality-Aware Dataflow;Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2;2023-01-27