Using Statistical Test Method to Establish a Decision Model of Performance Evaluation Matrix

Abstract

Many studies have pointed out that the Performance Evaluation Matrix (PEM) is a convenient and useful tool for the evaluation, analysis, and improvement of service operating systems. All service items of the operating system can collect customer satisfaction and importance through questionnaires and then convert them into satisfaction indices and importance indices to establish PEM and its evaluation rules. Since the indices have unknown parameters, if the evaluation is performed directly by the point estimates of the indices, there will be a risk of misjudgment due to sampling error. In addition, most of the studies only determine the critical-to-quality (CTQ) that needs to be improved, and do not discuss the treatment rules in the case of limited resources nor perform the confirmation after improvement. Therefore, to address similar research gaps, this paper proposed the unbiased estimators of these two indices and determined the critical-to-quality (CTQ) service items which need to be improved through the one-tailed statistical hypothesis test by building a PEM method of the satisfaction index. In addition, through the one-tailed statistical hypothesis test method of the importance index, the improvement priority of service items was determined under the condition of limited resources. Confirmation of the effect on improvement is an important step in management. Thus, this paper adopted a statistical two-tailed hypothesis test to verify whether the satisfaction of all the CTQ service items that need to be improved was enhanced. Since the method proposed in this paper was established through statistical hypothesis tests, the risk of misjudgment due to sampling error could be reduced. Obviously, reducing the misjudgment risk is the advantage of the method in this paper. Based on the precondition, utilizing the model in this study may assist the industries to determine CTQ rapidly, implement the most efficient improvement under the condition of limited resources and also confirm the improvement effect at the same time. Finally, a case study of computer-assisted language learning system (CALL System) was used to illustrate a way to apply the model proposed in this paper.