Predicting the effectiveness of the online clinical clerkship curriculum: Development of a multivariate prediction model and validation study

Abstract

Given scientific and technological advancements, expectations of online medical education are increasing. However, there is no way to predict the effectiveness of online clinical clerkship curricula. To develop a prediction model, we conducted cross-sectional national surveys in Japan. Social media surveys were conducted among medical students in Japan during the periods May–June 2020 and February–March 2021. We used the former for the derivation dataset and the latter for the validation dataset. We asked students questions in three areas: 1) opportunities to learn from each educational approach (lectures, medical quizzes, assignments, oral presentations, observation of physicians’ practice, clinical skills practice, participation in interprofessional meetings, and interactive discussions with physicians) in online clinical clerkships compared to face-to-face, 2) frequency of technical problems on online platforms, and 3) satisfaction and motivation as outcome measurements. We developed a scoring system based on a multivariate prediction model for satisfaction and motivation in a cross-sectional study of 1,671 medical students during the period May–June 2020. We externally validated this scoring with a cross-sectional study of 106 medical students during February–March 2021 and assessed its predictive performance. The final prediction models in the derivation dataset included eight variables (frequency of lectures, medical quizzes, oral presentations, observation of physicians’ practice, clinical skills practice, participation in interprofessional meetings, interactive discussions with physicians, and technical problems). We applied the prediction models created using the derivation dataset to a validation dataset. The prediction performance values, based on the area under the receiver operating characteristic curve, were 0.69 for satisfaction (sensitivity, 0.50; specificity, 0.89) and 0.75 for motivation (sensitivity, 0.71; specificity, 0.85). We developed a prediction model for the effectiveness of the online clinical clerkship curriculum, based on students’ satisfaction and motivation. Our model will accurately predict and improve the online clinical clerkship curriculum effectiveness.