Machine learning's model-agnostic interpretability on The Prediction of Students' Academic Performance in Video-Conference-Assisted Online Learning During the Covid-19 Pandemic

Abstract

Abstract The Covid-19 pandemic had an immediate impact on higher education. Although online technology has made contributions to higher education, its adoption has had a significant impact on learning activities during the Covid-19 pandemic. This paper proposed a predictive model for predicting students’ academic performance in video-conference-assisted online learning (VCAOL) during Covid-19 pandemic based on machine learning approach. We investigated: Random Forest (RF), Support Vector Machine (SVM) and Gaussian Naive Bayes (GNB). There were 361 data gathered as a dataset from September 2022 to January 2023. The overall result revealed RF outperformed SVM and GNB with accuracy score of 60.27%, precision 59.46%, recall 60.27%, F1-score 59.51% and ROC AUC 87%. Understanding a machine learning model's black-box output was crucial for providing predictions that explain why and how they were developed. SHAP value of global interpretability to visualize global feature importance revealed that students' performance while using VCAOL (Performance) was the most critical attribute for predicting students' academic performance. The SHAP local interpretability bar plot revealed that ‘student academic performance was still well achieved during the learning process, despite using video conferencing during the Covid-19 pandemic’ (Performance), when Performance decreased it contributed negative impact on students’ academic performance in VCAOL during Covid-19.