An Experimental Outlook on Quality Metrics for Process Modelling: A Systematic Review and Meta Analysis

Abstract

The ideology behind process modelling is to visualise lengthy event logs into simple representations interpretable to the end user. Classifying process models as simple or complex is based on criteria that evaluate attributes of models and quantify them on a scale. These metrics measure various characteristics of process models and describe their qualities. Over the years, vast amounts of metrics have been proposed in the community, making it difficult to find and select the appropriate ones for implementation. This paper presents a state-of-the-art meta-review that lists and summarises all the evaluation metrics proposed to date. We have studied the behaviour of the four most widely used metrics in process mining with an experiment. Further, we have used seven healthcare domain datasets of varying natures to analyse the behaviour of these metrics under different threshold conditions. Our work aims to propose and demonstrate the capabilities to use our selected metrics as a standard of measurement for the process mining domain.