A Practical Example of the Relevance of Computational Psychometric Experiments in Clinical Scale Validation

Abstract

At the core of effective clinical diagnostics using psychometric instruments relies to a strong specificity of the target psychological constructs. Recent computational methods thus hold promises to significantly advance psychometrical validation paradigms. Using DSM-5-Tr criteria, we developed a scale to identify high-functioning autism, which was then administered to 110 high-functioning autism patients and 110 control subjects. Relevant items were selected using multiple regression procedures, and its psychometric properties were evaluated through measures of internal consistency, factor analysis, and a comparative computational experiment using several XGBoost-type algorithms. However, although traditional metric measurements were satisfactory, the subsequent comparative computational experiment was unable to demonstrate significantly superior performance of models trained with the study scale compared to models trained on comorbidity scales. These findings underscored the importance of computational psychometrics in confirming that the constructs measured by clinical scales are specific to the conditions they are intended to distinguish, providing a critical control step that might constitute a new canon in psychometric validation procedures.