Essay Marking on-Screen: Implications for Assessment Validity

Abstract

Computer-assisted assessment offers many benefits over traditional paper methods. However, in transferring from one medium to another, it is crucial to ascertain the extent to which the new medium may alter the nature of traditional assessment practice or affect marking reliability. Whilst there is a substantial body of research comparing marking and marker behaviour on-screen and on paper, only a paucity of the available literature relates to the marking of extended responses. Research into on-screen assessment of continuous writing and its impact upon markers' judgements is, therefore, both timely and important. This article reports on the insights drawn from a series of onscreen marking trials at Cambridge Assessment which attempt to investigate marker reliability of extended responses, construct validity and whether factors such as annotation and navigation differentially influence marker performance across marking modes. The findings described here seek to ascertain whether markers make qualitatively different assessments when marking the same piece of writing but through a different medium. The article explains how the trial influenced the decision to move to on-screen marking of Checkpoint English and highlights the challenge of maximising ease of marking without compromising assessment validity.