Predicting Medical Student Performance With a Situational Judgment Test in Admissions

Abstract

Abstract Purpose To explore correlations between AAMC situational judgment test (SJT) scores, other admissions data, and learners’ medical school performance. Method First- and second-year medical students from 8 U.S. MD-granting medical schools completed a prototype version of the AAMC SJT in 2017. Outcomes included research-only faculty ratings of student performance, final course grades, and faculty evaluations of student performance, 2017–2018 and 2018–2019 academic years. Bivariate correlations were used to investigate the relationship between SJT scores and student performance outcomes and hierarchical regressions to investigate whether SJT scores provided incremental validity over MCAT total scores and cumulative undergraduate grade point averages (UGPAs) for predicting student performance outcomes. Results In general, there were small positive correlations with research-only faculty ratings from the first year of medical school, with the highest for social skills/service orientation (r corrected = .33, P < .05). Correlations were higher, with the highest for social skills/service orientation and cultural competence (r corrected = .33 and .36, respectively, P < .05) in the second year in medical school. SJT scores improved prediction of research-only faculty ratings over MCAT total scores and UGPAs for reliability and dependability/capacity for improvement, cultural competence, social skills/service orientation, and the overall composite score in the first year and for resilience and adaptability, social skills/service orientation, cultural competence, and the overall composite score in the second year. SJT scores demonstrated small correlations with course grades (r sample-weighted = .10, P = ns) and faculty evaluations related to professionalism skills (r sample-weighted = .14, P < .05); however, MCAT total scores explained most of the variance associated with course outcomes. Conclusions These studies provide initial evidence that SJT scores may add value to the medical school admissions process because scores were related to faculty ratings of professional behaviors and provided unique information relative to MCAT scores and UGPAs.