Item Retention as a Feature Selection Task: Developing Abbreviated Measures Using Shapley Values

Abstract

AbstractCreating abbreviated measures from lengthy questionnaires is important for reducing respondent burden while improving response quality. Though factor analytic strategies have been used to guide item retention for abbreviated questionnaires, item retention can be conceptualized as a feature selection task amenable to machine learning approaches. The present study tested a machine learning-guided approach to item retention, specifically item-level importance as measured by Shapley values for the prediction of total score, to create abbreviated versions of the Penn State Worry Questionnaire (PSWQ) in a sample of 3,906 secondary school students. Results showed that Shapley values were a useful measure for determining item retention in creating abbreviated versions of the PSWQ, demonstrating concordance with the full PSWQ. As item-level importance varied based on the proportion of the worry distribution predicted (e.g., high versus low PSWQ scores), item retention is dependent on the intended purpose of the abbreviated measure. Illustrative examples are presented.