Reliability and Quality of Online Multiple Mini interviews for Admissions in an MBBS program

Abstract

Abstract Background: MMI is a widely used method for assessing the non-cognitive skills of students, but the logistics and costs associated with organizing in-person MMI can be substantial. Virtual MMI, such as those conducted through platforms like WhatsApp Video calls, offer increased convenience, yet their reliability and quality remain uncertain. The objective of the study is to determine the reliability and quality (difficulty and discrimination indices) metrics of MMI scores conducted through WhatsApp Video call. Methods: Six MMI stations were used to assess the non-cognitive attributes of the students. In this descriptive study, the scores obtained by 678 students in MMI, F.Sc (equivalent of high school) and MDCAT were used to analyze the data. Item analysis was employed to assess quality of MMI stations. The reliability was calculated using Cronbach’s alpha and Pearson correlation (r) was performed between MDCAT scores, F.Sc. Scores and MMI scores to assess significant correlation. Results: The overall reliability of MMI in this study was 0.87 while the reliability for each interview station ranged between 0.92 - 0.95. The difficulty level of MMI stations ranged from easy (0.57) to moderately difficult (0.77). The discrimination index was found to be in the range of 0.53-0.78. Conclusion: The psychometric analysis of MMI scores demonstrated good reliability and quality (discrimination and difficulty index) with the stations showing acceptable discrimination and moderate difficulty. Hence, online multiple mini-interviews were found useful to assess non-cognitive skills for admission in MBBS program.