Addressing Class Imbalances in Video Time-Series Data for Estimation of Learner Engagement: “Over Sampling with Skipped Moving Average”

Abstract

Disengagement of students during online learning significantly impacts the effectiveness of online education. Thus, accurately estimating when students are not engaged is a critical aspect of online-learning research. However, the inherent characteristics of public datasets often lead to issues of class imbalances and data insufficiency. Moreover, the instability of video time-series data further complicates data processing in related research. Our research aims to tackle class imbalances and instability of video time-series data in estimating learner engagement, particularly in scenarios with limited data. In the present paper, we introduce “Skipped Moving Average”, an innovative oversampling technique designed to augment video time-series data representing disengaged students. Furthermore, we employ long short-term memory (LSTM) and long short-term memory fully convolutional network (LSTM-FCN) models to evaluate the effectiveness of our method and compare it to the synthetic minority over-sampling technique (SMOTE). This approach ensures a thorough evaluation of our method’s effectiveness in addressing video time-series data imbalances and in enhancing the accuracy of engagement estimation. The results demonstrate that our proposed method outperforms others in terms of both performance and stability across sequence deep learning models.