When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications-Reference-Cited by-同舟云学术

When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications

Published:2024-07-10 Issue: Volume:33 Page:1104-1114
ISSN:
Container-title:Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval
language:
Short-container-title:

Author:

Liu Qidong¹^ORCID,Wu Xian²^ORCID,Zhao Xiangyu³^ORCID,Zhu Yuanshao⁴^ORCID,Xu Derong⁵^ORCID,Tian Feng¹^ORCID,Zheng Yefeng²^ORCID

Affiliation:

1. Xi'an Jiaotong University, Xi'an, China

2. Tencent YouTu Lab, Jarvis Research Center, Shenzhen, China

3. City University of Hong Kong, Hong Kong, Hong Kong

4. Southern University of Science and Technology, Shenzhen, China

5. University of Science and Technology of China, Hefei, China

Funder

SIRG - CityU Strategic Interdisciplinary Research Grant

CCF-Tencent Open Fund

CityU - HKIDS Early Career Research Grant

APRC - CityU New Research Initiatives

Hong Kong Environmental and Conservation Fund

Research Impact Fund

Hong Kong ITC Innovation and Technology Fund Midstream Research Programme for Universities Project

Publisher

ACM

Link

https://dl.acm.org/doi/pdf/10.1145/3626772.3657722

Reference62 articles.

1. Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning

2. ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts

3. Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877--1901.

4. Preliminary study on the construction of Chinese medical knowledge graph;Byambasuren Odma;Journal of Chinese Information Processing,2019

5. Michael Crawshaw. 2020. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796 (2020).