The ELAPS framework: Experimental Linear Algebra Performance Studies

Abstract

In scientific computing, optimal use of computing resources comes at the cost of extensive coding, tuning, and benchmarking. While the classic approach of “features first, performance later” is supported by a variety of tools such as Tau, Vampir, and Scalasca, the emerging performance-centric approach, in which both features and performance are primary objectives, is still lacking suitable development tools. For dense linear algebra applications, we fill this gap with the Experimental Linear Algebra Performance Studies (ELAPS) framework, a multi-platform open-source environment for easy, fast, and yet powerful performance experimentation and prototyping. In contrast to many existing tools, ELAPS targets the beginning of the development process, assisting application developers in both algorithmic and optimization decisions. With ELAPS, users construct experiments to investigate how performance and efficiency depend on factors such as caching, algorithmic parameters, problem size, and parallelism. Experiments are designed either through Python scripts or a specialized Graphical User Interface (GUI), and run on a spectrum of architectures, ranging from laptops to accelerators and clusters. The resulting reports provide various metrics and statistics that can be analyzed both numerically and visually. In this article, we introduce ELAPS and illustrate its practical value in guiding critical performance decisions already in early development stages.