Building a Bigger Toolbox: The Construct Validity of Existing and Proposed Measures of Careless Responding to Cognitive Ability Tests

Abstract

Employers commonly use cognitive ability tests in the personnel selection process. Although ability tests are excellent predictors of job performance, their validity may be compromised when test takers engage in careless responding. It is thus important for researchers to have access to effective careless responding measures, which allow researchers to screen for careless responding and to evaluate efforts to prevent careless responding. Previous research has primarily used two types of measures to assess careless responding to ability tests—response time and self-reported carelessness. In the current paper, we expand the careless responding assessment toolbox by examining the construct validity of four additional measures: (1) infrequency, (2) instructed-response, (3) long-string, and (4) intra-individual response variability (IRV) indices. Expanding the available set of careless responding indices is important because the strengths of new indices may offset the weaknesses of existing indices and would allow researchers to better assess heterogeneous careless response behaviors. Across three datasets ( N = 1,193), we found strong support for the validity of the response-time and infrequency indices, and moderate support for the validity of the instructed-response and IRV indices.