Affiliation:
1. University of Luxembourg, Luxembourg
2. Luxembourg Institute of Science and Technology, Luxembourg
3. Singapore Management University, Singapore
Abstract
Applying deep learning (DL) to science is a new trend in recent years, which leads DL engineering to become an important problem. Although training data preparation, model architecture design, and model training are the normal processes to build DL models, all of them are complex and costly. Therefore, reusing the open-sourced pre-trained model is a practical way to bypass this hurdle for developers. Given a specific task, developers can collect massive pre-trained deep neural networks from public sources for reusing. However, testing the performance (e.g., accuracy and robustness) of multiple deep neural networks (DNNs) and recommending which model should be used is challenging regarding the scarcity of labeled data and the demand for domain expertise. In this article, we propose a labeling-free (LaF) model selection approach to overcome the limitations of labeling efforts for automated model reusing. The main idea is to statistically learn a Bayesian model to infer the models’ specialty only based on predicted labels. We evaluate LaF using nine benchmark datasets, including image, text, and source code, and 165 DNNs, considering both the accuracy and robustness of models. The experimental results demonstrate that LaF outperforms the baseline methods by up to 0.74 and 0.53 on Spearman’s correlation and Kendall’s τ, respectively.
Funder
Luxembourg National Research Funds
Publisher
Association for Computing Machinery (ACM)
Reference59 articles.
1. AOJ: Online Programming Challenge. 2018. AIZU online judge. Retrieved from https://judge.u-aizu.ac.jp/onlinejudge/. Accessed 10 January 2021.
2. Retrieved from github.com/Testing-Multiple-DL-Models/SDS/tree/ main/models 2021 DNN models for Fashion-MNIST
3. LaF project site. 2021. Project website of ranking multiple DNNs. Retrieved from https://sites.google.com/view/ranking-of-multiple-DNNs
4. Retrieved from 2022
5. MLOps. 2022. Machine Learning Model Operationalization Management . Retrieved from https://ml-ops.org/
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities;ACM Transactions on Software Engineering and Methodology;2024-08-26
2. Test Optimization in DNN Testing: A Survey;ACM Transactions on Software Engineering and Methodology;2024-04-20
3. DistXplore: Distribution-Guided Testing for Evaluating and Enhancing Deep Learning Systems;Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering;2023-11-30