Abstract
AbstractSystems that rely on Machine Learning (ML systems) have differing demands on quality—known as non-functional requirements (NFRs)—from traditional systems. NFRs for ML systems may differ in their definition, measurement, scope, and comparative importance. Despite the importance of NFRs in ensuring the quality ML systems, our understanding of all of these aspects is lacking compared to our understanding of NFRs in traditional domains. We have conducted interviews and a survey to understand how NFRs for ML systems are perceived among practitioners from both industry and academia. We have identified the degree of importance that practitioners place on different NFRs, including cases where practitioners are in agreement or have differences of opinion. We explore how NFRs are defined and measured over different aspects of a ML system (i.e., model, data, or whole system). We also identify challenges associated with NFR definition and measurement. Finally, we explore differences in perspective between practitioners in industry, academia, or a blended context. This knowledge illustrates how NFRs for ML systems are treated in current practice, and helps to guide future RE for ML efforts.
Publisher
Springer Science and Business Media LLC
Subject
Information Systems,Software
Reference58 articles.
1. Ali MA, Yap NK, Ghani AAA, Zulzalil H, Admodisastro NI, Najafabadi AA (2022) A systematic mapping of quality models for AI systems, software and components. Appl Sci 12(17):8700
2. Ameller D, Franch X, Gómez C, Martínez-Fernández S, Araújo J, Biffl S, Cabot J, Cortellessa V, Méndez D, Moreira A et al (2019) Dealing with non-functional requirements in model-driven development: a survey. IEEE Trans Softw Eng 1:1–2
3. Amershi S, Begel A, Bird C, DeLine R, Gall H, Kamar E, Nagappan N, Nushi B, Zimmermann T (2019) Software engineering for machine learning: a case study. In: 2019 IEEE/ACM 41st international conference on software engineering: software engineering in practice (ICSE-SEIP). IEEE, pp 291–300
4. Anisetti M, Ardagna CA, Damiani E, Panero PG (2020) A methodology for non-functional property evaluation of machine learning models. In: Proceedings of the 12th international conference on management of digital EcoSystems, pp 38–45
5. Khan Mohammad Habibullah and Jennifer Horkoff. Non-functional requirements for machine learning: understanding current use and challenges in industry. In: 2021 IEEE 29th International Requirements Engineering Conference (RE), pages 13–23. IEEE, 2021.
Cited by
7 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献