Compressed Collective Sparse-Sketch for Distributed Data-Parallel Training of Deep Learning Models-Reference-Cited by-同舟云学术

Compressed Collective Sparse-Sketch for Distributed Data-Parallel Training of Deep Learning Models

Published:2023-04 Issue:4 Volume:41 Page:941-963
ISSN:0733-8716
Container-title:IEEE Journal on Selected Areas in Communications
language:
Short-container-title:IEEE J. Select. Areas Commun.

Author:

Ge Keshi¹^ORCID,Lu Kai¹,Fu Yongquan¹^ORCID,Deng Xiaoge¹,Lai Zhiquan¹^ORCID,Li Dongsheng¹^ORCID

Affiliation:

1. College of Computer, National University of Defense Technology, Changsha, China

Funder

National Natural Science Foundation of China

National Key Research and Development Program of China

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Subject

Electrical and Electronic Engineering,Computer Networks and Communications

Link

http://xplorestaging.ieee.org/ielx7/49/10075675/10050389.pdf?arnumber=10050389

Reference59 articles.

1. PyTorch: An imperative style, high-performance deep learning library;paszke;Proc Adv Neural Inf Process Syst Annu Conf Neural Inf Process Syst (NeurIPS),2019

2. Advances and Open Problems in Federated Learning

3. TensorFlow: A system for large-scale machine learning;abadi;Proc of USENIX Symp on Operating Systems Design and Implementation (OSDI),2016

4. FetchSGD: Communication-efficient federated learning with sketching;rothchild;Proc 37th Int Conf Mach Learn (ICML),2020

5. QSGD: Communication-efficient SGD via gradient quantization and encoding;alistarh;Proc Adv Neural Inf Process Syst Annu Conf Neural Inf Process Syst (NeurIPS),2017

Cited by 3 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. A Multidimensional Communication Scheduling Method for Hybrid Parallel DNN Training;IEEE Transactions on Parallel and Distributed Systems;2024-08

2. An efficient bandwidth-adaptive gradient compression algorithm for distributed training of deep neural networks;Journal of Systems Architecture;2024-05

3. Guest Editorial Communication-Efficient Distributed Learning Over Networks;IEEE Journal on Selected Areas in Communications;2023-04