STR: Hybrid Tensor Re-Generation to Break Memory Wall for DNN Training-Reference-Cited by-同舟云学术

STR: Hybrid Tensor Re-Generation to Break Memory Wall for DNN Training

Author:

Zong Zan¹^ORCID,Lin Li²^ORCID,Lin Leilei³^ORCID,Wen Lijie⁴^ORCID,Sun Yu⁵^ORCID

Affiliation:

1. Department of Computer Science and Technology, Tsinghua University, Beijing, China

2. School of Computer Science and Engineering, Southeast University, Nanjing, Jiangsu, China

3. School of Management, Capital Normal University, Beijing, China

4. School of Software, Tsinghua University, Beijing, China

5. College of Computer Science, Nankai University, Tianjin, China

Funder

National Key Research and Development Program of China

National Natural Science Foundation of China

R&D Program of Beijing Municipal Education Commission

Natural Science Foundation of Tianjin City

Tsinghua BNRist and Beijing Key Laboratory of Industrial Big Data System and Application

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Subject

Computational Theory and Mathematics,Hardware and Architecture,Signal Processing

Link

Reference58 articles.

1. Dynamic tensor rematerialization;kirisame;Proc Int Conf Learn Representations,2021

2. Horovod: Fast and easy distributed deep learning in tensorflow;sergeev,2018

3. Training deep nets with sublinear memory cost;chen,2016

5. Dynamic memory management for GPU-based training of deep neural networks;shriram;Proc IEEE Int Parallel Distrib Process Symp,2019

Cited by 1 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. CAPTURE: Memory-Centric Partitioning for Distributed DNN Training with Hybrid Parallelism;2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics (HiPC);2023-12-18