The Comparability of Marking on Screen and on Paper: The Case of Liberal Studies in Hong Kong

Abstract

This article details an investigation into the onscreen marking (OSM) of Liberal Studies (LS) in Hong Kong—where paper-based marking (PBM) of public examinations is being phased out and wholly superseded by OSM. The study involved 14 markers who had previously rated Liberal Studies scripts on screen in the 2009 Hong Kong Advanced Level examination. In the study, the 14 markers re-marked, on paper, a number of the scripts they had marked on screen in the 2009 examination. Using multi-faceted Rasch analysis, a five-faceted design was employed, modeling markers, test takers, input questions, rating scales, and the marking medium. Results showed that all factors generally exhibited good data fit. With the major facet for investigation being the method of marking, logit values centred at zero emerged. The hypothesis that the method of marking does not intrude on scores awarded to test takers when scripts are marked on screen or on paper was therefore accepted. Since all public examinations in Hong Kong will be marked solely on screen in 2012, it is vital for professional and public confidence that the scores returned from the OSM marking system can be considered as reliable as those obtained from paper-based marking. Results from the current study suggest that this is so. Given that many countries and jurisdictions globally are considering the implementation of OSM technology—albeit not in the all-embracing manner of Hong Kong—the current study provides further validation for the worldwide adoption of such technology.