Understanding data storage and ingestion for large-scale deep recommendation model training-Reference-Cited by-同舟云学术

Understanding data storage and ingestion for large-scale deep recommendation model training

Published:2022-06-11 Issue: Volume: Page:
ISSN:
Container-title:Proceedings of the 49th Annual International Symposium on Computer Architecture
language:
Short-container-title:

Author:

Zhao Mark¹,Agarwal Niket¹,Basant Aarti¹,Gedik Buğra¹,Pan Satadru¹,Ozdal Mustafa¹,Komuravelli Rakesh¹,Pan Jerry¹,Bao Tianshu¹,Lu Haowei¹,Narayanan Sundaram¹,Langman Jack¹,Wilfong Kevin¹,Rastogi Harsha¹,Wu Carole-Jean¹,Kozyrakis Christos²,Pol Parik¹

Affiliation:

1. Meta

2. Stanford University

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3470496.3533044

Reference88 articles.

1. 2021. NVIDIA Data Loading Library (DALI). https://developer.nvidia.com/dali 2021. NVIDIA Data Loading Library (DALI). https://developer.nvidia.com/dali

2. 2022. Apache Arrow. https://arrow.apache.org/ 2022. Apache Arrow. https://arrow.apache.org/

3. 2022. Apache Avro. https://avro.apache.org/ 2022. Apache Avro. https://avro.apache.org/

4. 2022. Apache ORC. https://orc.apache.org/ 2022. Apache ORC. https://orc.apache.org/

5. 2022. Apache Parquet. https://parquet.apache.org/ 2022. Apache Parquet. https://parquet.apache.org/

Cited by 36 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Crux: GPU-Efficient Communication Scheduling for Deep Learning Training;Proceedings of the ACM SIGCOMM 2024 Conference;2024-08-04

2. A Selective Preprocessing Offloading Framework for Reducing Data Traffic in DL Training;Proceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems;2024-07-08

3. Can Storage Devices be Power Adaptive?;Proceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems;2024-07-08

4. PreSto: An In-Storage Data Preprocessing System for Training Recommendation Models;2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA);2024-06-29

5. The Case For Data Centre Hyperloops;2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA);2024-06-29