A review of two decades of research on the application of multicriteria decision making techniques to evaluate e-learning’s effectiveness

Abstract

Multicriteria decision-making (MCDM) techniques have been widely adopted to evaluate the effectiveness of e-learning. However, the literature review has not kept pace with the rapid accumulation of knowledge in this field. This study systematically reviews the MCDM techniques applied in e-learning issues. In total, we reviewed 77 published studies selected from the Web of Science and Scopus databases. We classified the selected studies by the publication year, the authors’ nationality, and the type of MCDM techniques examined. We further discovered that the majority of previous studies adopted Information System Success Model (D&M model) which was proposed by Delone and McLean in 1992. Due to limited features that were provided by the e-learning system back in the mid-2000, the original D&M model might not consider some significant factors such as students’ characteristics, instructors’ characteristics, user interface and learning community. The goals of this systematic literature review are to understand if the original D&M model is addressed in modern e-learning and to determine if new factors have emerged to evaluate e-learning’s effectiveness but not captured by the original D&M model. This review contributes three new theoretical perspectives. First, reclassification of the D&M model is conducted to include learners’ characteristics, instructors’ characteristics, user interface, and learning community. Second, this study discovers that there is a need to perform reclassification of overlapping subfactors. Third, this review also identifies that the dependencies among the factors in the D&M model are inadequately examined in previous studies. Researchers can utilize the findings of this study as a foundation to formulate their research frameworks, and practitioners can integrate the significant factors identified in this study into their decision-making processes when developing e-learning at their institutions.