Automated assembly of pre-equated language proficiency tests

Abstract

This study examines the effectiveness of an automated language proficiency test assembly system introduced at the Lackland Air Force Base Defense Language Institute English Language Center in several stages over the past year. Since it is desirable that the recently recalibrated item bank and newly implemented test assembly hardware and software be effective in the rapid generation of equivalent test forms, the criteria of effectiveness adopted for this study have been the established classical standards of test equivalence. Thus, the focus of the study is on the investigation of the equivalence of mean score difficulty, total score variance and intercorrelation covariance across ECL test forms. An experimental comparison of four randomly selected, computer-generated ECL test forms was made employing a representative sample of 338 ELC examinees. Results indicated that a high level of test-form equivalence and uniformly high internal consistency evidence exist across ECL test forms, exceeding the levels present in any known commer cially available language tests. It was concluded that the automated system meets requisite quality standards as implemented.