Adaptive Assessment and Content Recommendation in Online Programming Courses: On the Use of Elo-rating

Abstract

Online learning systems should support students preparedness for professional practice by equipping them with the necessary skills while keeping them engaged and active. In that regard, the development of online learning systems that support students’ development and engagement with programming is a challenging process. Early career computer science professionals are required not only to understand and master numerous programming concepts but also to efficiently learn how to apply them in different contexts. A prerequisite for an effective and engaging learning process is the existence of adaptive and flexible learning environments that are beneficial for both students and teachers. Students can benefit from personalized content adapted to their individual goals, knowledge, and needs; while teachers can be relieved from the pressure to uniformly and promptly evaluate hundreds of student assignments. This study proposes and puts into practice a method for evaluating learning content difficulty and students’ knowledge proficiency utilizing a modified Elo-rating method. The proposed method effectively pairs learning content difficulty with students’ proficiency, and creates personalized recommendations based on the generated ratings. The method was implemented in a programming tutoring system and tested with interactive learning content for object oriented-programming. By collecting quantitative and qualitative data from students who used the system for one semester, the findings reveal that the proposed method can generate recommendations that are relevant to students and has the potential to assist teachers in grading students by providing a more holistic understanding of their progress over time.