Beyond descriptive taxonomies in data analytics: a systematic evaluation approach for data-driven method pipelines

Abstract

AbstractTaxonomies can serve as a valuable tool to capture dimensions and characteristics of data analytics solutions in a structured manner and thus create transparency about different design options of the technical solution space. However, previous taxonomic approaches often remain at a purely descriptive level without leveraging morphological structures to investigate the mechanisms between different combinatorial options given in data analytics pipelines. To this end, we propose a taxonomic evaluation approach to evaluate and construct the technical core of analytical information systems more systematically. Specifically, we present a rough guidance model consisting of four steps, which we subsequently instantiate with two application scenarios from the fields of industrial maintenance and predictive business process monitoring. In this way, we demonstrate how taxonomic frameworks can guide the creation of structured evaluation studies to consider the construction and assessment of data analytics pipelines in a multi-perspective and holistic manner. Our approach is sufficiently generic to be applied to various domains, scenarios, and decision support tasks.