Visual Tuning

Author:

Yu Bruce X.B.1ORCID,Chang Jianlong2ORCID,Wang Haixin3ORCID,Liu Lingbo4ORCID,Wang Shijie2ORCID,Wang Zhiyu2ORCID,Lin Junfan4ORCID,Xie Lingxi2ORCID,Li Haojie5ORCID,Lin Zhouchen6ORCID,Tian Qi2ORCID,Chen Chang Wen7ORCID

Affiliation:

1. Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University, Haining, China and Zhejiang Provincial Engineering Research Center for Multimodal Transport Logistics Large Models, Haining China

2. Huawei, Shenzhen China

3. Peking University, National Engineering Research Center for Software Engineering, Beijing China

4. Peng Cheng Laboratory, Shenzhen China

5. Shandong University of Science and Technology, College of Computer Science and Engineering, Qingdao China

6. National Key Lab of General AI, Peking University, School of Intelligence Science and Technology, Peking University, Beijing, China and Pazhou Laboratory (Huangpu), Guangzhou China

7. The Hong Kong Polytechnic University, Department of Computing, Hong Kong, Hong Kong

Abstract

Fine-tuning visual models has been widely shown promising performance on many downstream visual tasks. With the surprising development of pre-trained visual foundation models, visual tuning jumped out of the standard modus operandi that fine-tunes the whole pre-trained model or just the fully connected layer. Instead, recent advances can achieve superior performance than full-tuning the whole pre-trained parameters by updating far fewer parameters, enabling edge devices and downstream applications to reuse the increasingly large foundation models deployed on the cloud. With the aim of helping researchers get the full picture and future directions of visual tuning, this survey characterizes a large and thoughtful selection of recent works, providing a systematic and comprehensive overview of existing work and models. Specifically, it provides a detailed background of visual tuning and categorizes recent visual tuning techniques into five groups: fine-tuning, prompt tuning, adapter tuning, parameter tuning, and remapping tuning. Meanwhile, it offers some exciting research directions for prospective pre-training and various interactions in visual tuning.

Funder

National Key R&D Program of China

National Natural Science Foundation of China

Huawei Technologies

Publisher

Association for Computing Machinery (ACM)

Reference301 articles.

1. Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr Yana Hasson Karel Lenc Arthur Mensch Katie Millican Malcolm Reynolds et al. 2022. Flamingo: A visual language model for few-shot learning. Proc. NeurIPS 35 (2022) 23716–23736.

2. Saad Albawi, Tareq Abed Mohammed, and Saad Al-Zawi. 2017. Understanding of a convolutional neural network. In Proceedings of the ICET. IEEE, 1–6.

3. Reinald Kim Amplayo, Kang Min Yoo, and Sang-Woo Lee. 2022. Attribute injection for pretrained language models: A new benchmark and an efficient method. In Proceedings of the COLING. 1051–1064.

4. Simone Angarano Francesco Salvetti Mauro Martini and Marcello Chiaberge. 2023. Generative adversarial superresolution at the edge with knowledge distillation. Engineering Applications of Artificial Intelligence 123 (2023) 106407.

5. Anubhav Ashok, Nicholas Rhinehart, Fares Beainy, and Kris M. Kitani. 2018. N2n learning: Network to network compression via policy gradient reinforcement learning. In Proceedings of the ICLR.

Cited by 3 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. EGCN++: A New Fusion Strategy for Ensemble Learning in Skeleton-Based Rehabilitation Exercise Assessment;IEEE Transactions on Pattern Analysis and Machine Intelligence;2024-09

2. Deep Learning for Abnormal Human Behavior Detection in Surveillance Videos—A Survey;Electronics;2024-06-30

3. Parameter-efficient framework for surgical action triplet recognition;International Journal of Computer Assisted Radiology and Surgery;2024-04-30

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3