Abstract
AbstractThe purpose of this paper is to apply the Bookmark method to the standard setting. Based on the Rasch Model in item response theory, a ninth-grade mathematics achievement test in china has been taken as an example of the standard setting, and 2 cut scores have been established to distinguish students into different performance levels eventually, namely basic and proficient cut scores. In addition, based on the use of generalizability theory, the standard error of the cut scores and the practical standard error are used as indicators to explore the effect that panelists and the standard setting rounds have made on the precision of Bookmark standard setting results through a mixed design of (p: g) × r. Result shows that the cut scores of basic and proficient were respectively 52.25 and 67.53. Besides, increasing the number of panelists in the group or standard setting rounds will reduce the standard error of the cut scores and the practical standard error. In addition, practical standard error is a necessary reference index when applying generalizability theory to analyze the cut scores established by Bookmark method, while the standard error of cut scores also has a great reference value.
Funder
the Ministry of Education Foundation of the People’s Republic of China,
the Philosophy and Social Science Foundation of Guangdong Province
the Natural Science Foundation of Guangdong Province
Publisher
Springer Science and Business Media LLC
Reference27 articles.
1. Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). American Council on Education.
2. Brennan, R. L. (2000). Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement, 24(4), 339–353.
3. Brennan, R. L., & Lockwood, R. E. (1980). A comparison of the Nedelsky and Angoff cutting score procedures using generalizability theory. Applied Psychological Measurement, 4(2), 219–240.
4. Buckendahl, C. W., Davis, S. L., Plake, B. S., Sireci, S. G., Hambleton, R. K., Zenisky, A. L., & Wells, C. S. (2009). Evaluation of the National Assessment of Educational Progress: Final report. U.S. Department of Education.
5. Buckendahl, C. W., Smith, R. W., Impara, J. C., & Plake, B. S. (2002). A comparison of Angoff and Bookmark standard setting methods. Journal of Educational Measurement, 39(3), 253–263.