Machine Learning Software and Pipelines

Abstract

AbstractThe use of the term “pipeline” in modern applications of machine learning, and more in general of statistical computing, is interesting because of its dual meaning: in software engineering, it denotes the process of developing and delivering software; in data science, it denotes the sequence of steps required to prepare, analyze, and draw conclusions from data.Machine learning pipelines combine aspects of both definitions because they involve the process of building and operating the software infrastructure to develop and use machine learning models as well as the data analysis steps that are implemented by those models. In this article, we discuss how modern practices from software engineering and data science combine to shape machine learning software in an iterative workflow defined by the interplay of code and data.