OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning-Reference-Cited by-同舟云学术

OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning

Published:2024-08-24 Issue: Volume:33 Page:6137-6147
ISSN:
Container-title:Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
language:
Short-container-title:

Author:

Ye Rui¹^ORCID,Wang Wenhao²^ORCID,Chai Jingyi¹^ORCID,Li Dihan³^ORCID,Li Zexi²^ORCID,Xu Yinda¹^ORCID,Du Yaxin¹^ORCID,Wang Yanfeng⁴^ORCID,Chen Siheng⁴^ORCID

Affiliation:

1. Shanghai Jiao Tong University, Shanghai, China

2. Zhejiang University, Zhejiang, China

3. University of Southern California, Los Angeles, USA

4. Shanghai Jiao Tong University & Shanghai AI Laboratory, Shanghai, China

Funder

NSFC

Science and Technology Commission of Shanghai Municipality

National Key R&D Program of China

Science and Technology Commission of Shanghai Municipal

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3637528.3671582

Reference95 articles.

1. Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen Nova DasSarma Dawn Drain Stanislav Fort Deep Ganguli Tom Henighan et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022).

2. Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell Jackson Kernion Andy Jones Anna Chen Anna Goldie Azalia Mirhoseini Cameron McKinnon et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 (2022).

3. Language models are few-shot learners;Brown Tom;NIPS,2020

4. Chaochao Chen, Xiaohua Feng, Jun Zhou, Jianwei Yin, and Xiaolin Zheng. 2023. Federated large language model: A position paper. arXiv preprint arXiv:2307.08925 (2023).

5. Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman Alex Ray Raul Puri Gretchen Krueger Michael Petrov Heidy Khlaaf Girish Sastry Pamela Mishkin Brooke Chan Scott Gray Nick Ryder Mikhail Pavlov Alethea Power Lukasz Kaiser Mohammad Bavarian Clemens Winter Philippe Tillet Felipe Petroski Such Dave Cummings Matthias Plappert Fotios Chantzis Elizabeth Barnes Ariel Herbert-Voss William Hebgen Guss Alex Nichol Alex Paino Nikolas Tezak Jie Tang Igor Babuschkin Suchir Balaji Shantanu Jain William Saunders Christopher Hesse Andrew N. Carr Jan Leike Josh Achiam Vedant Misra Evan Morikawa Alec Radford Matthew Knight Miles Brundage Mira Murati Katie Mayer Peter Welinder Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. arxiv: 2107.03374 [cs.LG]