Automated Assessment in Programming Courses: A Case Study during the COVID-19 Era

Abstract

The COVID-19 pandemic imposed in many countries, in the short term, the interruption of face-to-face teaching activities and, in the medium term, the existence of a ‘new normal’, in which teaching methods should be able to switch from face-to-face to remote overnight. However, this flexibility can pose a great difficulty, especially in the assessment of practical courses with a high student–teacher ratio, in which the assessment tools or methods used in face-to-face learning are not ready to be adopted within a fully online environment. This article presents a case study describing the transformation of the assessment method of a programming course in higher education to a fully online format during the COVID-19 pandemic, by means of an automated student-centered assessment tool. To evaluate the new assessment method, we studied students’ interactions with the tool, as well as students’ perceptions, which were measured with two different surveys: one for the programming assignments and one for the final exam. The results show that the students’ perceptions of the assessment tool were highly positive: if using the tool had been optional, the majority of them would have chosen to use it without a doubt, and they would like other courses to involve a tool like the one presented in this article. A discussion about the use of this tool in subsequent years in the same and related courses is also presented, analyzing the sustainability of this new assessment method.