Affiliation:
1. Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, Nanjing 211100, China
2. School of Computer and Information, Hohai University, Nanjing 211100, China
Abstract
In wireless distributed computing systems, worker nodes connect to a master node wirelessly and perform large-scale computational tasks that are parallelized across them. However, the common phenomenon of straggling (i.e., worker nodes often experience unpredictable slowdown during computation and communication) and packet losses due to severe channel fading can significantly increase the latency of computational tasks. In this paper, we consider a heterogeneous, wireless, distributed computing system performing large-scale matrix multiplications which form the core of many machine learning applications. To address the aforementioned challenges, we first propose a random linear network coding (RLNC) approach that leverages the linearity of matrix multiplication, which has many salient properties, including ratelessness, maximum straggler tolerance and near-ideal load balancing. We then theoretically demonstrate that its latency converges to the optimum in probability when the matrix size grows to infinity. To combat the high encoding and decoding overheads of the RLNC approach, we further propose a practical variation based on batched sparse (BATS) code. The effectiveness of our proposed approaches is demonstrated by numerical simulations.
Funder
the Water Conservancy Project of Jiangsu Province
the National Natural Science Foundation of China
the Fundamental Research Funds for the Central Universities
the Natural Science Foundation of Jiangsu Province
the Future Network Scientific Research Fund Project
Subject
General Physics and Astronomy
Reference37 articles.
1. Advances and open problems in federated learning;Kairouz;Found. Trends Mach. Learn.,2021
2. Drolia, U., Guo, K., and Narasimhan, P. (2017, January 12–14). Precog: Prefetching for image recognition applications at the edge. Proceedings of the Second ACM/IEEE Symposium on Edge Computing, San Jose, CA, USA.
3. Wireless distributed computing: A survey of research challenges;Datla;IEEE Commun. Mag.,2012
4. A scalable framework for wireless distributed computing;Li;IEEE-ACM Trans. Netw.,2017
5. The tail at scale;Dean;Commun. ACM,2013