Proficiency descriptors based on a scale-anchoring study of the new TOEFL iBT reading test

Abstract

Providing information to test takers and test score users about the abilities of test takers at different score levels has been a persistent problem in educational and psychological measurement (Carroll, 1993). Since the 1990s Educational Testing Service has been investigating solutions to this problem through the development of proficiency scaling procedures and questiondifficulty research. In 1997 a proficiency scale was developed for the Test of English as a Foreign Language (TOEFL) Reading Comprehension section using a tree-based regression approach. The current study describes a scale-anchoring study of the new TOEFL iBT reading test and the resulting proficiency descriptors that are now part of the TOEFL iBT score report. The goal was to provide descriptive information about the abilities that test takers need in order to answer questions correctly. These abilities are those articulated in the new TOEFL Reading Framework and in the guidelines for writing test questions. Scale anchoring is a method of creating descriptors of the performance of test takers that is based on both empirical data and judgments by test developers. It has been used with a variety of assessments, including the National Assessment of Educational Progress (NAEP) and the Trends in International Mathematics and Science Study (TIMSS).