Affiliation:
1. Department of Information Physics and Computing, Graduate School of Information Science and Technology, The University of Tokyo, Tokyo 113-8656, Japan
Abstract
Matrix multiplication is important in various information-processing applications, including the computation of eigenvalues and eigenvectors, and in combinatorial optimization algorithms. Therefore, reducing the computation time of matrix products is essential to speed up scientific and practical calculations. Several approaches have been proposed to speed up this process, including GPUs, fast matrix multiplication libraries, custom hardware, and efficient approximate matrix multiplication (AMM) algorithms. However, research to date has yet to focus on accelerating AMMs for general matrices on GPUs, despite the potential of GPUs to perform fast and accurate matrix product calculations. In this paper, we propose a method for improving Monte Carlo AMMs. We also give an analytical solution for the optimal values of the hyperparameters in the proposed method. The proposed method improves the approximation of the matrix product without increasing the computation time compared to the conventional AMMs. It is also designed to work well with parallel operations on GPUs and can be incorporated into various algorithms. Finally, the proposed method is applied to a power method used for eigenvalue computation. We demonstrate that, on an NVIDIA A100 GPU, the computation time can be halved compared to the conventional power method using cuBLAS.
Funder
CREST project
Japan Science and Technology Agency
Transformative Research Areas
Subject
General Physics and Astronomy
Reference39 articles.
1. Learning representations by back-propagating errors;Rumelhart;Nature,1986
2. Sarle, W. (1994, January 10–13). Neural Networks and Statistical Models. Proceedings of the 19th Annual SAS Users Group International Conference, Dallas, TX, USA.
3. Learning the parts of objects by non-negative matrix factorization;Lee;Nature,1999
4. Binary optimization by momentum annealing;Okuyama;Phys. Rev. E,2019
5. Bottou, L. (1999). On-Line Learning in Neural Networks, Cambridge University Press.