Development of a Computerized Adaptive Testing System for Assessing Social Knowledge in People With Schizophrenia

Abstract

Abstract Importance: A psychometrically sound measure of social knowledge (SK) is necessary to assess people with schizophrenia because they tend to have moderate to severe deficits in SK. Objective: To develop a computerized adaptive test (CAT) for assessing SK in people with schizophrenia. Design: Two phases, consisting of (1) development and validation of an SK item bank and (2) determination of the best stopping rules for the CAT. Setting: Two psychiatric hospitals. Participants: Two hundred thirty-six people diagnosed with schizophrenia through convenience sampling. Measure: Computerized Adaptive Test–Social Knowledge (CAT–SK). Results: The SK items were examined using Rasch analysis. A CAT simulation was performed to determine the best set of stopping rules for achieving high reliability and efficiency. After unsuitable items were removed, 71 items remained with acceptable model fit (infit and outfit mean square <1.4) and no gender bias. Two suboptimal alternative sets of rules were identified. The most efficient set used 21 items to achieve acceptable Rasch reliability (.81). The most reliable set used 40 items to achieve satisfactory Rasch reliability (.88). High correlations (r > .93) between CAT–SK scores and scores on the SK item bank support the concurrent validity of the CAT–SK. Conclusions and Relevance: The CAT–SK appears to be a valid assessment that can provide reliable or efficient measures of SK. If high reliability is needed, examiners can adopt the most reliable set of 40 items. If efficiency is the primary concern, they can adopt the most efficient set of 21 items. What This Article Adds: The CAT–SK is a valid measure of SK with flexibility to meet examiners’ needs.