A novel criterion based on continuous ranked probability score for performance evaluation of control charts

Abstract

AbstractRun length distributions are generally used to characterize the performance of a control chart in signaling alarms when a process is out‐of‐control. Since it is usually difficult to directly compare distributions, statistics of the run length distribution are commonly adopted as the performance criteria in practice. Particularly, the average run length (ARL) and its extended versions play a dominant role. However, due to the skewness of the run length distribution, the ARL cannot accurately reflect the central tendency and may be misleading in some cases. In order to comprehensively summarize the information of the run length distribution, a novel criterion is proposed based on the continuous ranked probability score (CRPS). The CRPS‐based criterion measures the difference between the run length distribution and the ideal constant value 0 for the run length. It has advantages of easy computation and good interpretability. Furthermore, theoretical properties and geometric representation guarantee that the CRPS‐based criterion is statistically consistent, informative of both first and second moments of the run length distribution, and robust to extreme values. Results of numerical experiments show that the proposed criterion favors control charts with higher probability to detect outliers earlier, and is a superior metric for characterizing the run length distribution.