Affiliation:
1. University of Science and Technology of China, Hefei, Anhui, China
2. Rutgers University, Newark, New Jersey, USA
Abstract
Labeling data (e.g., labeling the people, objects, actions, and scene in images) comprehensively and efficiently is a widely needed but challenging task. Numerous models were proposed to label various data and many approaches were designed to enhance the ability of deep learning models or accelerate them. Unfortunately, a single machine-learning model is not powerful enough to extract various semantic information from data. Given certain applications, such as image retrieval platforms and photo album management apps, it is often required to execute a collection of models to obtain sufficient labels. With limited computing resources and stringent delay, given a data stream and a collection of applicable resource-hungry deep-learning models, we design a novel approach to adaptively schedule a subset of these models to execute on each data item, aiming to maximize the value of the model output (e.g., the number of high-confidence labels). Achieving this lofty goal is nontrivial since a model’s output on any data item is content-dependent and unknown until we execute it. To tackle this, we propose an
Adaptive Model Scheduling
framework, consisting of (1) a deep reinforcement learning-based approach to predict the value of unexecuted models by mining semantic relationship among diverse models, and (2) two heuristic algorithms to adaptively schedule the model execution order under a deadline or deadline-memory constraints, respectively. The proposed framework does not require any prior knowledge of the data, which works as a powerful complement to existing model optimization technologies. We conduct extensive evaluations on five diverse image datasets and 30 popular image labeling models to demonstrate the effectiveness of our design: our design could save around 53% execution time without loss of any valuable labels.
Funder
National Key R&D Program of China
China National Natural Science Foundation
Key Research Program of Frontier Sciences, CAS
Publisher
Association for Computing Machinery (ACM)
Reference38 articles.
1. Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. OpenFace: A general-purpose face recognition library with mobile applications. Technical Report. CMU-CS-16-118, CMU School of Computer Science.
2. Zhe Cao Gines Hidalgo Tomas Simon Shih-En Wei and Yaser Sheikh. 2017. OpenPose: Realtime multi-person 2D pose estimation using part Affinity fields. In Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition . 7291–7299.
3. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
4. Compressing Convolutional Neural Networks in the Frequency Domain
5. Q-Learning and SARSA: A comparison between two intelligent stochastic control approaches for financial trading;Corazza Marco;Research Paper Series No. 15. University Ca’Foscari of Venice, Dept. of Economics,2015
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Kernel Fisher Dictionary Transfer Learning;ACM Transactions on Knowledge Discovery from Data;2023-05-12
2. Efficient Deep Ensemble Inference via Query Difficulty-dependent Task Scheduling;2023 IEEE 39th International Conference on Data Engineering (ICDE);2023-04
3. InFi;Proceedings of the 28th Annual International Conference on Mobile Computing And Networking;2022-10-14
4. Quality-aided Annotation Service Selection in MLaaS Market;2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS);2022-06-10
5. COCA: Cost-Effective Collaborative Annotation System by Combining Experts and Amateurs;2022 IEEE 38th International Conference on Data Engineering (ICDE);2022-05