Software-hardware co-design for fast and scalable training of deep learning recommendation models-Reference-Cited by-同舟云学术

Software-hardware co-design for fast and scalable training of deep learning recommendation models

Published:2022-06-11 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 49th Annual International Symposium on Computer Architecture
language:
Short-container-title:

Author:

Mudigere Dheevatsa¹,Hao Yuchen¹,Huang Jianyu¹,Jia Zhihao²,Tulloch Andrew¹,Sridharan Srinivas¹,Liu Xing¹,Ozdal Mustafa¹,Nie Jade¹,Park Jongsoo¹,Luo Liang¹,Yang Jie (Amy)¹,Gao Leon¹,Ivchenko Dmytro¹,Basant Aarti¹,Hu Yuxi¹,Yang Jiyan¹,Ardestani Ehsan K.¹,Wang Xiaodong¹,Komuravelli Rakesh¹,Chu Ching-Hsiang¹,Yilmaz Serhat¹,Li Huayu¹,Qian Jiyuan¹,Feng Zhuobo¹,Ma Yinbin¹,Yang Junjie¹,Wen Ellie¹,Li Hong¹,Yang Lin¹,Sun Chonglin¹,Zhao Whitney¹,Melts Dimitry¹,Dhulipala Krishna¹,Kishore KR¹,Graf Tyler¹,Eisenman Assaf¹,Matam Kiran Kumar¹,Gangidi Adi¹,Chen Guoqiang Jerry¹,Krishnan Manoj¹,Nayak Avinash¹,Nair Krishnakumar¹,Muthiah Bharath¹,khorashadi Mahmoud¹,Bhattacharya Pallab¹,Lapukhov Petr¹,Naumov Maxim¹,Mathews Ajit¹,Qiao Lin¹,Smelyanskiy Mikhail¹,Jia Bill¹,Rao Vijay¹

Affiliation:

1. Meta Platforms

2. Carnegie Mellon University

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3470496.3533727

Reference71 articles.

1. [n.d.]. NVIDIA Collective Communications Library (NCCL) https://developer.nvidia.com/nccl. [n.d.]. NVIDIA Collective Communications Library (NCCL) https://developer.nvidia.com/nccl.

2. [n.d.]. OCP Open rack standard (v2) https://www.opencompute.org/wiki/Open_Rack/SpecsAndDesigns#RACK_Standards. [n.d.]. OCP Open rack standard (v2) https://www.opencompute.org/wiki/Open_Rack/SpecsAndDesigns#RACK_Standards.

3. Martín Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dandelion Mané Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Viégas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org. Martín Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dandelion Mané Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Viégas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.

4. Bilge Acun Matthew Murphy Xiaodong Wang Jade Nie Carole-Jean Wu and Kim Hazelwood. 2020. Understanding Training Efficiency of Deep Learning Recommendation Models at Scale. arXiv:2011.05497 [cs.AR] Bilge Acun Matthew Murphy Xiaodong Wang Jade Nie Carole-Jean Wu and Kim Hazelwood. 2020. Understanding Training Efficiency of Deep Learning Recommendation Models at Scale. arXiv:2011.05497 [cs.AR]

5. Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. arXiv:2005.14165 [cs.CL] Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. arXiv:2005.14165 [cs.CL]

Cited by 29 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Flexible silicon photonic architecture for accelerating distributed deep learning;Journal of Optical Communications and Networking;2024-01-09

2. FPGA-Accelerated Data Preprocessing for Personalized Recommendation Systems;IEEE Computer Architecture Letters;2024-01

3. High-bandwidth density silicon photonic resonators for energy-efficient optical interconnects;Applied Physics Reviews;2023-11-15

4. Aurelia: CXL Fabric with Tentacle;Proceedings of the 4th Workshop on Resource Disaggregation and Serverless;2023-10-23

5. UGACHE: A Unified GPU Cache for Embedding-based Deep Learning;Proceedings of the 29th Symposium on Operating Systems Principles;2023-10-23