Assessing Response Quality by Using Multivariate Control Charts for Numerical and Categorical Response Quality Indicators

Abstract

Abstract When assessing interview response quality to identify potentially low-quality interviews, both numerical and categorical response quality indicators (mixed indicators) are usually available. However, research on how to use them simultaneously is very rare. In the current article, we extend the application of conventional multivariate control charts to include response quality indicators that are of a mixed type. We analyze data from the eighth round of the European Social Survey in Belgium, characterized by six numerical and two categorical response quality indicators. First, we employ a principal component analysis mix procedure (PCA Mix) to transform the mixed quality indicators into principal components. The principal component scores are subsequently used to construct a Hotelling T2 statistic. To deal with the non-multivariate normal nature of the principal component scores obtained from the PCA Mix, a nonparametric bootstrap method is then applied to calculate the control limit for the T2 statistic. Second, we suggest tools to interpret an identified outlier in terms of finding the responsible original indicator(s). Third, we present a cyclic procedure for determining the “in-control” data, by iteratively removing the outliers until the process is considered as being in control. Lastly, we identify the most important indicators that discriminate the outliers from the in-control data. Our results imply that multivariate control charts based on relevant projection tools such as PCA Mix in combination with the bootstrap technique have great potential for use in evaluating interview response quality and identifying outliers.