Affiliation:
1. Georgia Institute of Technology
Abstract
Three-dimensional (3D)-stacking technology and the memory-wall problem have popularized processing-in-memory (PIM) concepts again, which offers the benefits of bandwidth and energy savings by offloading computations to functional units inside the memory. Several memory vendors have also started to integrate computation logics into the memory, such as Hybrid Memory Cube (HMC), the latest version of which supports up to 18 in-memory atomic instructions. Although industry prototypes have motivated studies for investigating efficient methods and architectures for PIM, researchers have not proposed a systematic way for identifying the benefits of
instruction-level PIM offloading
. As a result, compiler support for recognizing offloading candidates and utilizing instruction-level PIM offloading is unavailable. In this article, we analyze the advantages of instruction-level PIM offloading in the context of HMC-atomic instructions for graph-computing applications and propose
CAIRO
, a compiler-assisted technique and decision model for enabling instruction-level offloading of PIM without any burden on programmers. To develop CAIRO, we analyzed how instruction offloading enables performance gain in both CPU and GPU workloads. Our studies show that performance gain from bandwidth savings, the ratio of number of cache misses to total cache accesses, and the overhead of host atomic instructions are the key factors in selecting an offloading candidate. Based on our analytical models, we characterize the properties of beneficial and nonbeneficial candidates for offloading. We evaluate CAIRO with 27 multithreaded CPU and 36 GPU benchmarks. In our evaluation, CAIRO not only doubles the speedup for a set of PIM-beneficial workloads by exploiting HMC-atomic instructions but also prevents slowdown caused by incorrect offloading decisions for other workloads.
Funder
National Science Foundation
Publisher
Association for Computing Machinery (ACM)
Subject
Hardware and Architecture,Information Systems,Software
Cited by
33 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Architecture-Aware Currying;2023 32nd International Conference on Parallel Architectures and Compilation Techniques (PACT);2023-10-21
2. Processing-in-Memory Development Strategy for AI Computing Using Main-Path and Doc2Vec Analyses;Sustainability;2023-08-16
3. TransPimLib: Efficient Transcendental Functions for Processing-in-Memory Systems;2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS);2023-04
4. PIMFlow: Compiler and Runtime Support for CNN Models on Processing-in-Memory DRAM;Proceedings of the 21st ACM/IEEE International Symposium on Code Generation and Optimization;2023-02-17
5. SAPIVe: Simple AVX to PIM Vectorizer;2022 XII Brazilian Symposium on Computing Systems Engineering (SBESC);2022-11-21