The Development of the Attitude towards Computerized Assessment Scale

Abstract

Test equivalence can be evaluated in terms of four aspects: psychometric, behavioral, experiential, and individual differences (i.e., relativity of equivalence) (Honaker, 1988). This study examined the psychometric properties of the Attitude Towards Computerized Assessment Scale (ATCAS) designed to assess one of these criteria, namely, experiential equivalence. Specifically, the ATCAS was developed to explore examinees' emotional, perceptual and attitudinal reactions towards computerized relative to conventional testing methods. The validity and reliability of the ATCAS is addressed by presenting research findings from two independent studies. An exploratory factor analysis (EFA) of the intercorrelations of responses on the 13-items comprising the ATCAS was conducted in Study 1 using data from 92 undergraduate psychology students. The results of the EFA yielded two qualitatively different aspects of the individual's perceptions and experience of computer-based testing (CBT): Ease of Use and CBT-Confidence. The second study used data collected from an Australian Defence Force (ADF) sample to further explore and then confirm the factor structure of the ATCAS. The second study was divided into two phases. Based on the responses of 201 ADF applicants, an EFA of the ATCAS supported the presence of the Ease of Use and CBT-Confidence factors. Based on the responses of 331 ADF applicants, a confirmatory factor analysis (CFA) of ATCAS items supported the two-factor model. In studies 1 and 2, internal consistency was evidenced for the ATCAS and its subscales with values ranging from .76 to .92. Implications for the study of experiential equivalence are discussed, while areas for future research are also considered.