Language testing: towards a methodology for the study of the method effect

Abstract

This article discusses the methodology for investigating the effect of item format on test performance. While correlational procedures have traditionally been used in examinations of the method effect in the context of norm-referenced tests, the signifi cance of differences between means is more important for criterion-referenced testing because test diffi culty has a direct influence on classification decisions. Special attention is given to ways of controlling confounding factors in repeated-measures experimental designs.