Systematic evaluation of Business Process Management Systems

Abstract

Selecting a Business Process Management Systems (BPMS) for an organization requires a thorough evaluation of its capabilities considering the whole support of the business process lifecycle and the organizational environment in which the BPMS will be used. In a previous work, we have proposed a methodology for the systematic evaluation of BPMS, ensuring the quality of the results and the repeatability of the evaluation process. The methodology envisions a quantitative and qualitative evaluation regarding the fulfillment of key features that BPMS must provide in the context of a given organization. However, it was mostly focused on required functional and non-technical aspects. In this paper, we present the extension of our methodology with a detailed definition of non-functional aspects to be evaluated, a set of test cases for their evaluation, and the development of a case study as a complimentary qualitative evaluation. We also performed a fine tuning of the methodology based on a comprehensive comparison with other existent methodologies and the provision of tool support. To illustrate the approach, we present results from the evaluation of open source and proprietary BPMS which constitute both a validation and assessment of our proposal and a contribution to knowledge regarding the capacities of selected BPMS technologies.