1. Bommasani Rishi 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258
2. Known unknowns: indirect energy effects of information and communication technology
3. Decentralized training of foundation models in heterogeneous environments;Advances in Neural Information Processing Systems,2022
4. Narayanan, Deepak, 2021. Efficient large-scale language model training on gpu clusters using megatron-lm. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
5. Conglong Li Zhewei Yao 2022. DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing. https://www.microsoft.com/en-us/research/publication/deepspeed-data-efficiency-improving-deep-learning-model-quality-and-training-efficiency-via-efficient-data-sampling-and-routing/