Effectiveness of Feedback Based on Log File Analysis in Introductory Programming Courses

Abstract

Especially in large courses, feedback is often given only on the final results; less attention is paid to the programming process. Today, however, some programming environments, e.g., Thonny, log activities during programming and have the functionality of replaying the programming process. This information can be used to provide feedback, and this knowledge can be integrated into practical sessions in the classroom. This study aimed to analyse how feedback based on logs affects exam results, task completion time, the number of runs, error messages, and pastes (of the whole group and beginners and non-beginners separately). An experiment was conducted in 2020 and 2021 in the course “Introduction to Programming”. Some groups received additional feedback on homework throughout the course based on log information; the remaining groups worked as usual. Based on the information received from the logs, general recommendations were also offered in the practical sessions. Our study showed that feedback based on logs improved mainly exam test results and programming task solving time among beginners. Therefore, it would be a good method to use, especially in beginner groups.